Search

Deep learning-based methods have been widely researched in the areas of language and vision, demonstrating their capacity to understand long sequences of data and their usefulness in numerous helio-physics applications. Foundation models (FMs), which are pre-trained on a large-scale datasets, form the basis for a variety of downstream tasks. These models, especially those based on transformers in vision and language, show exceptional potential for adapting to a wide range of downstream applications. In this paper, we provide our perspective on the criteria for designing an FM for heliophysics and associated challenges and applications using the Solar Dynamics Observatory (SDO) dataset. We believe that this is the first study to design an FM in the domain of heliophysics., Comment: 31 Pages, 12 figures

`Extreme Classification'' (or XC) is the task of annotating data points (queries) with relevant labels (documents), from an extremely large set of $L$ possible labels, arising in search and recommendations. The most successful deep learning paradigm that has emerged over the last decade or so for XC is to embed the queries (and labels) using a deep encoder (e.g. DistilBERT), and use linear classifiers on top of the query embeddings. This architecture is of appeal because it enables millisecond-time inference using approximate nearest neighbor search (ANNS). The key question is how do we design training algorithms that are accurate as well as scale to $O(100M)$ labels on a limited number of GPUs. State-of-the-art XC techniques that demonstrate high accuracies (e.g., DEXML, Ren\'ee, DEXA) on standard datasets have per-epoch training time that scales as $O(L)$ or employ expensive negative sampling strategies, which are prohibitive in XC scenarios. In this work, we develop an accurate and scalable XC algorithm ASTRA with two key observations: (a) building ANNS index on the classifier vectors and retrieving hard negatives using the classifiers aligns the negative sampling strategy to the loss function optimized; (b) keeping the ANNS indices current as the classifiers change through the epochs is prohibitively expensive while using stale negatives (refreshed periodically) results in poor accuracy; to remedy this, we propose a negative sampling strategy that uses a mixture of importance sampling and uniform sampling. By extensive evaluation on standard XC as well as proprietary datasets with 120M labels, we demonstrate that ASTRA achieves SOTA precision, while reducing training time by 4x-15x relative to the second best.

As large language models (LLMs) evolve to handle increasingly longer contexts, serving inference requests for context lengths in the range of millions of tokens presents unique challenges. While existing techniques are effective for training, they fail to address the unique challenges of inference, such as varying prefill and decode phases and their associated latency constraints - like Time to First Token (TTFT) and Time Between Tokens (TBT). Furthermore, there are no long context inference solutions that allow batching requests to increase the hardware utilization today. In this paper, we propose three key innovations for efficient interactive long context LLM inference, without resorting to any approximation: adaptive chunking to reduce prefill overheads in mixed batching, Sequence Pipeline Parallelism (SPP) to lower TTFT, and KV Cache Parallelism (KVP) to minimize TBT. These contributions are combined into a 3D parallelism strategy, enabling Mnemosyne to scale interactive inference to context lengths at least up to 10 million tokens with high throughput enabled with batching. To our knowledge, Mnemosyne is the first to be able to achieve support for 10 million long context inference efficiently, while satisfying production-grade SLOs on TBT (30ms) on contexts up to and including 10 million.

Lattice gauge theories (LGTs) can be employed to understand a wide range of phenomena, from elementary particle scattering in high-energy physics to effective descriptions of many-body interactions in materials. Studying dynamical properties of emergent phases can be challenging as it requires solving many-body problems that are generally beyond perturbative limits. We investigate the dynamics of local excitations in a $\mathbb{Z}_2$ LGT using a two-dimensional lattice of superconducting qubits. We first construct a simple variational circuit which prepares low-energy states that have a large overlap with the ground state; then we create particles with local gates and simulate their quantum dynamics via a discretized time evolution. As the effective magnetic field is increased, our measurements show signatures of transitioning from deconfined to confined dynamics. For confined excitations, the magnetic field induces a tension in the string connecting them. Our method allows us to experimentally image string dynamics in a (2+1)D LGT from which we uncover two distinct regimes inside the confining phase: for weak confinement the string fluctuates strongly in the transverse direction, while for strong confinement transverse fluctuations are effectively frozen. In addition, we demonstrate a resonance condition at which dynamical string breaking is facilitated. Our LGT implementation on a quantum processor presents a novel set of techniques for investigating emergent particle and string dynamics.

Artificial Intelligence (AI) has demonstrated significant potential in automating various medical imaging tasks, which could soon become routine in clinical practice for disease diagnosis, prognosis, treatment planning, and post-treatment surveillance. However, the privacy concerns surrounding patient data present a major barrier to the widespread adoption of AI in medical imaging, as large, diverse training datasets are essential for developing accurate, generalizable, and robust Artificial intelligence models. Federated Learning (FL) offers a solution that enables organizations to train AI models collaboratively without sharing sensitive data. federated learning exchanges model training information, such as gradients, between the participating sites. Despite its promise, federated learning is still in its developmental stages and faces several challenges. Notably, sensitive information can still be inferred from the gradients shared during model training. Quantifying AI models' uncertainty is vital due to potential data distribution shifts post-deployment, which can affect model performance. Uncertainty quantification (UQ) in FL is particularly challenging due to data heterogeneity across participating sites. This review provides a comprehensive examination of FL, privacy-preserving FL (PPFL), and UQ in FL. We identify key gaps in current FL methodologies and propose future research directions to enhance data privacy and trustworthiness in medical imaging applications., Comment: 21 pages, 5 figures, 4 tables, Review paper, preprint to Radiology AI. arXiv admin note: text overlap with arXiv:2406.12815

Triggered by the realization that AI emulators can rival the performance of traditional numerical weather prediction models running on HPC systems, there is now an increasing number of large AI models that address use cases such as forecasting, downscaling, or nowcasting. While the parallel developments in the AI literature focus on foundation models -- models that can be effectively tuned to address multiple, different use cases -- the developments on the weather and climate side largely focus on single-use cases with particular emphasis on mid-range forecasting. We close this gap by introducing Prithvi WxC, a 2.3 billion parameter foundation model developed using 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture, incorporating concepts from various recent transformer models to effectively capture both regional and global dependencies in the input data. The model has been designed to accommodate large token counts to model weather phenomena in different topologies at fine resolutions. Furthermore, it is trained with a mixed objective that combines the paradigms of masked reconstruction with forecasting. We test the model on a set of challenging downstream tasks namely: Autoregressive rollout forecasting, Downscaling, Gravity wave flux parameterization, and Extreme events estimation. The pretrained model with 2.3 billion parameters, along with the associated fine-tuning workflows, has been publicly released as an open-source contribution via Hugging Face.

Ultra-luminous X-ray sources (ULXs) are high-mass X-ray binaries with an X-ray luminosity above $10^{39}$ erg s$^{-1}$. These ULXs can be powered by black holes that are more massive than $20M_\odot$, accreting in a standard regime, or lighter compact objects accreting supercritically. There are only a few ULXs with known optical or UV counterparts, and their nature is debated. Determining whether optical/UV radiation is produced by the donor star or by the accretion disc is crucial for understanding ULX physics and testing massive binary evolution. We conduct, for the first time, a fully consistent multi-wavelength spectral analysis of a ULX and its circumstellar nebula. We aim to establish the donor star type and test the presence of strong disc winds in the prototypical ULX Holmberg II X-1 (Ho II X-1). We intent to obtain a realistic spectral energy distribution of the ionising source, which is needed for robust nebula analysis. We acquired new UV spectra of Ho II X-1 with the HST and complemented them with archival optical and X-ray data. We explored the spectral energy distribution of the source and analysed the spectra using the stellar atmosphere code PoWR and the photoionisation code Cloudy. Our analysis of the X-ray, UV, and optical spectra of Ho II X-1 and its nebula consistently explains the observations. We do not find traces of disc wind signatures in the UV and the optical, rejecting previous claims of the ULX being a supercritical accretor. The optical/UV counterpart of HoII X-1 is explained by a B-type supergiant donor star. Thus, the observations are fully compatible with Ho II X-1 being a close binary consisting of an $\gtrsim 66\,M_\odot$ black hole accreting matter from an $\simeq 22 M_\odot$ B-supergiant companion. Also, we propose a possible evolution scenario for the system, suggesting that Ho II X-1 is a potential gravitational wave source progenitor., Comment: Accepted in A&A. 14 pages (12 main body + 2 appendix), 6 figures, 6 tables

Evaluating models and datasets in computer vision remains a challenging task, with most leaderboards relying solely on accuracy. While accuracy is a popular metric for model evaluation, it provides only a coarse assessment by considering a single model's score on all dataset items. This paper explores Item Response Theory (IRT), a framework that infers interpretable latent parameters for an ensemble of models and each dataset item, enabling richer evaluation and analysis beyond the single accuracy number. Leveraging IRT, we assess model calibration, select informative data subsets, and demonstrate the usefulness of its latent parameters for analyzing and comparing models and datasets in computer vision.

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical Concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48,000 medical problems and drug events and 10,000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction (IE) models such as BERT, Flan-T5, Llama3, and GPT-4 using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, LLMs offer no performance advantage over the smaller BERT models. The results emphasize the need for annotated training data to optimize models. Multiple fine-tuned transformer models achieved performance comparable to IAA for several extraction tasks., Comment: This is a pre-copy-editing, author-produced PDF of an article accepted for publication in JAMIA following peer review. The definitive publisher-authenticated version is available online at https://academic.oup.com/jamia/advance-article/doi/10.1093/jamia/ocae231/7748302

Ethanolamine (NH2CH2CH2OH) has recently been identified in the molecular cloud G+0.693-0.027, situated in the SgrB2 complex in the Galactic center. However, its presence in other regions, and in particular in star-forming sites, is still elusive. Given its likely role as a precursor to simple amino acids, understanding its presence in the star-forming region is required. Here, we present the experimentally obtained temperature-dependent spectral features and morphological behavior of pure ethanolamine ices under astrochemical conditions in the 2 - 12 micro meter (MIR) and 120 - 230 nm (VUV) regions for the first time. These features would help in understanding its photochemical behavior. In addition, we present the first chemical models specifically dedicated to ethanolamine. These models include all the discussed chemical routes from the literature, along with the estimated binding energies and activation energies from quantum chemical calculations reported in this work. We have found that surface reactions: CH2OH + NH2CH2 --> NH2CH2CH2OH and NH2 + C2H4OH --> NH2CH2CH2OH in warmer regions (60-90 K) could play a significant role in the formation of ethanolamine. Our modeled abundance of ethanolamine complements the upper limit of ethanolamine column density estimated in earlier observations in hot core/corino regions. Furthermore, we provide a theoretical estimation of the rotational and distortional constants for various species (such as HNCCO, NH2CHCO, and NH2CH2CO) related to ethanolamine that have not been studied in existing literature. This study could be valuable for identifying these species in the future., Comment: 18 pages, 6 figures. Accepted for the publication in The Astrophysical Journal

Quantum error correction provides a path to reach practical quantum computing by combining multiple physical qubits into a logical qubit, where the logical error rate is suppressed exponentially as more qubits are added. However, this exponential suppression only occurs if the physical error rate is below a critical threshold. In this work, we present two surface code memories operating below this threshold: a distance-7 code and a distance-5 code integrated with a real-time decoder. The logical error rate of our larger quantum memory is suppressed by a factor of $\Lambda$ = 2.14 $\pm$ 0.02 when increasing the code distance by two, culminating in a 101-qubit distance-7 code with 0.143% $\pm$ 0.003% error per cycle of error correction. This logical memory is also beyond break-even, exceeding its best physical qubit's lifetime by a factor of 2.4 $\pm$ 0.3. We maintain below-threshold performance when decoding in real time, achieving an average decoder latency of 63 $\mu$s at distance-5 up to a million cycles, with a cycle time of 1.1 $\mu$s. To probe the limits of our error-correction performance, we run repetition codes up to distance-29 and find that logical performance is limited by rare correlated error events occurring approximately once every hour, or 3 $\times$ 10$^9$ cycles. Our results present device performance that, if scaled, could realize the operational requirements of large scale fault-tolerant quantum algorithms., Comment: 10 pages, 4 figures, Supplementary Information

Current approaches to automated code generation often rely on monolithic models that lack real-time adaptability and scalability. This limitation is particularly evident in complex programming tasks that require dynamic adjustment and efficiency. The integration of neuroscience principles into Natural Language Processing (NLP) has the potential to revolutionize automated code generation. This paper presents CortexCompile, a novel modular system inspired by the specialized functions of the human brain's cortical regions. By emulating the distinct roles of the Prefrontal Cortex, Parietal Cortex, Temporal Lobe, and Motor Cortex, CortexCompile achieves significant advancements in scalability, efficiency, and adaptability compared to traditional monolithic models like GPT-4o. The system's architecture features a Task Orchestration Agent that manages dynamic task delegation and parallel processing, facilitating the generation of highly accurate and optimized code across increasingly complex programming tasks. Experimental evaluations demonstrate that CortexCompile consistently outperforms GPT-4o in development time, accuracy, and user satisfaction, particularly in tasks involving real-time strategy games and first-person shooters. These findings underscore the viability of neuroscience-inspired architectures in addressing the limitations of current NLP models, paving the way for more efficient and human-like AI systems., Comment: 17 pages, 6 figures

Deep Generative Models are frequently used to learn continuous representations of complex data distributions using a finite number of samples. For any generative model, including pre-trained foundation models with Diffusion or Transformer architectures, generation performance can significantly vary across the learned data manifold. In this paper we study the local geometry of the learned manifold and its relationship to generation outcomes for a wide range of generative models, including DDPM, Diffusion Transformer (DiT), and Stable Diffusion 1.4. Building on the theory of continuous piecewise-linear (CPWL) generators, we characterize the local geometry in terms of three geometric descriptors - scaling ($\psi$), rank ($\nu$), and complexity/un-smoothness ($\delta$). We provide quantitative and qualitative evidence showing that for a given latent-image pair, the local descriptors are indicative of generation aesthetics, diversity, and memorization by the generative model. Finally, we demonstrate that by training a reward model on the local scaling for Stable Diffusion, we can self-improve both generation aesthetics and diversity using `geometry reward' based guidance during denoising., Comment: Accepted for publication at ICLR 2025

We introduce Imagen 3, a latent diffusion model that generates high quality images from text prompts. We describe our quality and responsibility evaluations. Imagen 3 is preferred over other state-of-the-art (SOTA) models at the time of evaluation. In addition, we discuss issues around safety and representation, as well as methods we used to minimize the potential harm of our models.

Behavior cloning is a common imitation learning paradigm. Under behavior cloning the robot collects expert demonstrations, and then trains a policy to match the actions taken by the expert. This works well when the robot learner visits states where the expert has already demonstrated the correct action; but inevitably the robot will also encounter new states outside of its training dataset. If the robot learner takes the wrong action at these new states it could move farther from the training data, which in turn leads to increasingly incorrect actions and compounding errors. Existing works try to address this fundamental challenge by augmenting or enhancing the training data. By contrast, in our paper we develop the control theoretic properties of behavior cloned policies. Specifically, we consider the error dynamics between the system's current state and the states in the expert dataset. From the error dynamics we derive model-based and model-free conditions for stability: under these conditions the robot shapes its policy so that its current behavior converges towards example behaviors in the expert dataset. In practice, this results in Stable-BC, an easy to implement extension of standard behavior cloning that is provably robust to covariate shift. We demonstrate the effectiveness of our algorithm in simulations with interactive, nonlinear, and visual environments. We also conduct experiments where a robot arm uses Stable-BC to play air hockey. See our website here: https://collab.me.vt.edu/Stable-BC/

The projected sensitivity of the effective electron neutrino-mass measurement with the KATRIN experiment is below 0.3 eV (90 % CL) after five years of data acquisition. The sensitivity is affected by the increased rate of the background electrons from KATRIN's main spectrometer. A special shifted-analysing-plane (SAP) configuration was developed to reduce this background by a factor of two. The complex layout of electromagnetic fields in the SAP configuration requires a robust method of estimating these fields. We present in this paper a dedicated calibration measurement of the fields using conversion electrons of gaseous $^\mathrm{83m}$Kr, which enables the neutrino-mass measurements in the SAP configuration., Comment: 19 pages, 11 figures

Metal-poor massive stars drive the evolution of low-mass galaxies, both locally and at high redshift. However, quantifying the feedback they impart to their local surroundings remains uncertain because models of stellar evolution, mass loss, and ionizing spectra are unconstrained by observations below 20% solar metallicity ($Z_\odot$). We present new Keck Cosmic Web Imager optical spectroscopy of three O stars in the nearby dwarf galaxies Leo P, Sextans A, and WLM, which have gas-phase oxygen abundances of 3-14% $Z_\odot$. To characterize their fundamental stellar properties and radiation-driven winds, we fit PoWR atmosphere models to the optical spectra simultaneously with Hubble Space Telescope far-ultraviolet (FUV) spectra and multi-wavelength photometry. We find that all three stars have effective temperatures consistent with their spectral types and surface gravities typical of main-sequence dwarf stars. Yet, the combination of those inferred parameters and luminosity for the two lower-$Z$ stars is not reproduced by stellar evolution models, even those that include rotation or binary interactions. The scenario of multiple-star systems is difficult to reconcile with all available data, suggesting that these observations pose a challenge to current evolution models. We highlight the importance of validating the relationship between stellar mass, temperature, and luminosity at very low $Z$ for accurate estimates of ionizing photon production and spectral hardness. Finally, all three stars' FUV wind profiles reveal low mass-loss rates and terminal wind velocities in tension with expectations from widely adopted radiation-driven wind models. These results provide empirical benchmarks for future development of mass-loss and evolution models for metal-poor stellar populations., Comment: Accepted at ApJ. 10 figures, 25 pages

The smoothed particle hydrodynamics (SPH) method has been widely used to simulate incompressible and slightly compressible fluid flows. Adaptive refinement strategies to dynamically increase the resolution of the particles to capture sharp gradients in the flow have also been developed. However, most of the SPH schemes in the literature are not second-order convergent (SOC). Both second-order convergence and adaptive resolution are considered grand challenge problems in the SPH community. In this paper, we propose, for the first time, a second-order convergent (SOC) adaptive refinement strategy along with a SOC weakly-compressible SPH scheme. We employ the method of manufactured solutions to systematically develop the scheme and validate the solver. We demonstrate the order of convergence of the entire scheme using the Taylor-Green vortex problem and then go on to demonstrate the applicability of the method to simulate flow past a circular cylinder.

Surveys in the Milky Way and Large Magellanic Cloud revealed that the majority of massive stars will interact with companions during their lives. However, knowledge of the binary properties of massive stars at low metallicity, which approaches the conditions of the Early Universe, remains sparse. We present the Binarity at LOw Metallicity (BLOeM) campaign - an ESO large programme designed to obtain 25 epochs of spectroscopy for 929 massive stars in the SMC - the lowest metallicity conditions in which multiplicity is probed to date (Z = 0.2 Zsun). BLOeM will provide (i) the binary fraction, (ii) the orbital configurations of systems with periods P < 3 yr, (iii) dormant OB+BH binaries, and (iv) a legacy database of physical parameters of massive stars at low metallicity. The stars are observed with the LR02 setup of the giraffe instrument of the Very Large Telescope (3960-4570A, resolving power R=6200; typical signal-to-noise ratio S/N=70-100). This paper utilises the first 9 epochs obtained over a three-month time. We describe the survey and data reduction, perform a spectral classification of the stacked spectra, and construct a Hertzsprung-Russell diagram of the sample via spectral-type and photometric calibrations. The sample covers spectral types from O4 to F5, spanning the effective temperature and luminosity ranges 6.5

Context. B supergiants (BSGs) represent an important connection between the main sequence and more extreme evolutionary stages of massive stars. Additionally, lying toward the cool end of the hot star regime, determining their wind properties is crucial to constrain the evolution and feedback of massive stars as, for instance, they might manifest the bi-stability jump phenomenon. Aims. We undertake a detailed analysis of a representative sample of 18 Small Magellanic Cloud (SMC) BSGs within the ULLYSES and XShootU datasets. Our UV and optical analysis spans BSGs from B0 to B8 - covering the bi-stability jump region. We aim to evaluate their evolutionary status and verify what their wind properties say about the bi-stability jump in a low-metallicity environment. Methods. We used the CMFGEN to model the spectra and photometry (from UV to infrared) of our sample. We compare our results with different evolutionary models, with previous determinations in the literature of OB stars, and with diverging mass-loss recipes at the bi-stability jump. Additionally, we provide the first BSG models in the SMC including X-rays. Results. (i) Within a single-stellar evolution framework, the evolutionary status of early BSGs seem less clear than that of late BSGs, which agree with H-shell burning models. (ii) UV analysis shows evidence that BSGs contain X-rays in their atmospheres, for which we provide constraints. In general, we find higher X-ray luminosity (close to the standard log(L_X/L) ~ -7) for early BSGs. For cooler BSGs, lower values are preferred, log(L_X/L) ~ -8.5. (iii) The obtained mass-loss rates suggest neither a jump nor a monotonic decrease with temperature. Instead, a rather constant trend is observed, which is at odds with the increase found for Galactic BSGs. (iv) The wind velocity behavior with temperature shows a sharp drop at ~19 kK, similar to what is observed for Galactic BSGs., Comment: 33 pages (23+10)+(22 at Zenodo), 34 figures (21+13)+(21 at Zenodo), 7 tables (3+4)+(1 at Zenodo), accepted for publication

The ability to create superlattices in van der Waals (vdW) heterostructures via moir\'e interference heralded a new era in the science and technology of two-dimensional materials. Through precise control of the twist angle, flat bands and strongly correlated phases have been engineered. The precise twisting of vdW layers is in some sense a bottom-up approach--a single parameter can dial in a wide range of periodic structures. Here, we describe a top-down approach to engineering nanoscale potentials in vdW layers using a buried programmable ferroelectric layer. Ultra-low-voltage electron beam lithography (ULV-EBL) is used to program ferroelectric domains in a ferroelectric Al_{1-x}B_{x}N thin film through a graphene/hexagonal boron nitride (hBN) heterostructure that is transferred on top. We demonstrate ferroelectric field effects by creating a lateral p-n junction, and demonstrate spatial resolution down to 35 nm, limited by the resolution of our scanned probe characterization methods. This innovative, resist-free patterning method is predicted to achieve 10 nm resolution and enable arbitrary programming of vdW layers, opening a pathway to create new phases that are inaccessible by moir\'e techniques. The ability to "paint" different phases of matter on a single vdW "canvas" provides a wealth of new electronic and photonic functionalities., Comment: 9 pages, 4 figures and supplemental material

Published research highlights the presence of demographic bias in automated facial attribute classification. The proposed bias mitigation techniques are mostly based on supervised learning, which requires a large amount of labeled training data for generalizability and scalability. However, labeled data is limited, requires laborious annotation, poses privacy risks, and can perpetuate human bias. In contrast, self-supervised learning (SSL) capitalizes on freely available unlabeled data, rendering trained models more scalable and generalizable. However, these label-free SSL models may also introduce biases by sampling false negative pairs, especially at low-data regimes 200K images) under low compute settings. Further, SSL-based models may suffer from performance degradation due to a lack of quality assurance of the unlabeled data sourced from the web. This paper proposes a fully self-supervised pipeline for demographically fair facial attribute classifiers. Leveraging completely unlabeled data pseudolabeled via pre-trained encoders, diverse data curation techniques, and meta-learning-based weighted contrastive learning, our method significantly outperforms existing SSL approaches proposed for downstream image classification tasks. Extensive evaluations on the FairFace and CelebA datasets demonstrate the efficacy of our pipeline in obtaining fair performance over existing baselines. Thus, setting a new benchmark for SSL in the fairness of facial attribute classification., Comment: 13 pages, IJCB 2024

When Large Language Models (LLMs) are compressed using techniques such as quantization, the predominant way to demonstrate the validity of such techniques is by measuring the model's accuracy on various benchmarks.If the accuracies of the baseline model and the compressed model are close, it is assumed that there was negligible degradation in quality.However, even when the accuracy of baseline and compressed model are similar, we observe the phenomenon of flips, wherein answers change from correct to incorrect and vice versa in proportion.We conduct a detailed study of metrics across multiple compression techniques, models and datasets, demonstrating that the behavior of compressed models as visible to end-users is often significantly different from the baseline model, even when accuracy is similar.We further evaluate compressed models qualitatively and quantitatively using MT-Bench and show that compressed models are significantly worse than baseline models in this free-form generative task.Thus, we argue that compression techniques should also be evaluated using distance metrics.We propose two such metrics, KL-Divergence and flips, and show that they are well correlated.

Buckling instabilities driven by tissue growth underpin key developmental events such as the folding of the brain. Tissue growth is disordered due to cell-to-cell variability, but the effects of this variability on buckling are unknown. Here, we analyse what is perhaps the simplest setup of this problem: the buckling of an elastic rod with fixed ends driven by spatially varying growth. Combining analytical calculations for simple growth fields and numerical sampling of random growth fields, we show that variability can increase as well as decrease the growth threshold for buckling, even when growth variability does not cause any residual stresses. For random growth, we find that the shift of the buckling threshold correlates with spatial moments of the growth field. Our results imply that biological systems can either trigger or avoid buckling by exploiting the spatial arrangement of growth variability., Comment: 6 pages, 3 figures; Supplemental Material: 7 pages, 1 figure

Serving large language models (LLMs) in production can incur substantial costs, which has prompted recent advances in inference system optimizations. Today, these systems are evaluated against conventional latency and throughput metrics (eg. TTFT, TBT, Normalised Latency and TPOT). However, these metrics fail to fully capture the nuances of LLM inference, leading to an incomplete assessment of user-facing performance crucial for real-time applications such as chat and translation. In this paper, we first identify the pitfalls of current performance metrics in evaluating LLM inference systems. We then propose Etalon, a comprehensive performance evaluation framework that includes fluidity-index -- a novel metric designed to reflect the intricacies of the LLM inference process and its impact on real-time user experience. Finally, we evaluate various existing open-source platforms and model-as-a-service offerings using Etalon, discussing their strengths and weaknesses. Etalon is available at https://github.com/project-etalon/etalon.

In order to develop robots that can effectively serve as versatile and capable home assistants, it is crucial for them to reliably perceive and interact with a wide variety of objects across diverse environments. To this end, we proposed Open Vocabulary Mobile Manipulation as a key benchmark task for robotics: finding any object in a novel environment and placing it on any receptacle surface within that environment. We organized a NeurIPS 2023 competition featuring both simulation and real-world components to evaluate solutions to this task. Our baselines on the most challenging version of this task, using real perception in simulation, achieved only an 0.8% success rate; by the end of the competition, the best participants achieved an 10.8\% success rate, a 13x improvement. We observed that the most successful teams employed a variety of methods, yet two common threads emerged among the best solutions: enhancing error detection and recovery, and improving the integration of perception with decision-making processes. In this paper, we detail the results and methodologies used, both in simulation and real-world settings. We discuss the lessons learned and their implications for future research. Additionally, we compare performance in real and simulated environments, emphasizing the necessity for robust generalization to novel settings.

We present CLAMP-ViT, a data-free post-training quantization method for vision transformers (ViTs). We identify the limitations of recent techniques, notably their inability to leverage meaningful inter-patch relationships, leading to the generation of simplistic and semantically vague data, impacting quantization accuracy. CLAMP-ViT employs a two-stage approach, cyclically adapting between data generation and model quantization. Specifically, we incorporate a patch-level contrastive learning scheme to generate richer, semantically meaningful data. Furthermore, we leverage contrastive learning in layer-wise evolutionary search for fixed- and mixed-precision quantization to identify optimal quantization parameters while mitigating the effects of a non-smooth loss landscape. Extensive evaluations across various vision tasks demonstrate the superiority of CLAMP-ViT, with performance improvements of up to 3% in top-1 accuracy for classification, 0.6 mAP for object detection, and 1.5 mIoU for segmentation at similar or better compression ratio over existing alternatives. Code is available at https://github.com/georgia-tech-synergy-lab/CLAMP-ViT.git, Comment: ECCV 2024

Proton and $\alpha$-particle energy spectra were measured in coincidence with fission fragments at different relative angles in $^{16}$O (96 MeV) + $^{232}$Th reaction. The multiplicity spectra were analyzed within the framework of a Moving Source Disentangling Analysis (MSDA) to determine contributions from different emission stages. The MSDA conclusively shows ``Near Scission Emission (NSE)" as an essential component in the multiplicity spectra. In contrast to NSE $\alpha$ particles which emit mainly perpendicular (``equatorial emission"), the NSE protons are observed to be emitted perpendicular as well as parallel (``polar emission") to the fission axis with similar intensities ($\sim$20\% for each). Thus, around 40\% of total pre-scission protons are emitted near the scission stage, whereas the same fraction for $\alpha$ particles is only around 10\%. The inevitable presence of ``polar" and ``equatorial" NSE protons in a heavy-ion induced fission has been observed for the first time. Present results open up a new avenue to study the heavy-ion induced fission dynamics.

CONTEXT: The spectral analysis of hot, massive stars is a fundamental astrophysical method to obtain their intrinsic properties and their feedback. Quantitative spectroscopy for hot, massive stars requires detailed numerical modeling of the atmosphere and an iterative treatment to obtain the best solution within a given framework. AIMS: We present an overview of different techniques for the quantitative spectroscopy of hot stars employed within the X-Shooting ULLYSES collaboration, from grid-based approaches to tailored fits. By performing a blind test, we gain an overview about the similarities and differences of the resulting parameters. Our study aims to provide an overview of the parameter spread caused by different approaches. METHODS: For three different stars from the sample (SMC O5 star AzV 377, LMC O7 star Sk -69 50, and LMC O9 star Sk -66 171), we employ different atmosphere codes (CMFGEN, Fastwind, PoWR) and strategies to determine their best-fitting model. For our analyses, UV and optical spectra are used to derive the properties with some methods relying purely on optical data for comparison. To determine the overall spectral energy distribution, we further employ additional photometry from the literature. RESULTS: Effective temperatures for each of three sample stars agree within 3 kK while the differences in log g can be up to 0.2 dex. Luminosity differences of up to 0.1 dex result from different reddening assumptions, which seem to be larger for the methods employing a genetic algorithm. All sample stars are nitrogen-enriched. CONCLUSIONS: We find a reasonable agreement between the different methods. Tailored fitting tends to be able to minimize discrepancies obtained with more course or automatized treatments. UV spectral data is essential for the determination of realistic wind parameters. For one target (Sk -69 50), we find clear indications of an evolved status., Comment: 19+15 pages, 21+4 figures, accepted version (A&A 689, A30) including language editing, condensed abstract