Search

Finetuning is a common practice widespread across different communities to adapt pretrained models to particular tasks. Text classification is one of these tasks for which many pretrained models are available. On the other hand, ensembles of neural networks are typically used to boost performance and provide reliable uncertainty estimates. However, ensembling pretrained models for text classification is not a well-studied avenue. In this paper, we present a metadataset with predictions from five large finetuned models on six datasets, and report results of different ensembling strategies from these predictions. Our results shed light on how ensembling can improve the performance of finetuned text classifiers and incentivize future adoption of ensembles in such tasks., Comment: Workshop on Fine-Tuning in Modern Machine Learning @ NeurIPS 2024. arXiv admin note: text overlap with arXiv:2410.04520

In integrated power and gas energy system optimization models (ESOMs), pipeline gas transmission with linepack is a particularly complex problem due to its non-linear and non-convex character. For ESOMs based on mixed-integer linear programing, piecewise linearization is a well-established convexification approach for this problem, which, however, requires binary variables to model feasible combinations of linear gas flow and pressure segments and thus can quickly become computationally challenging. In order to improve computational performance, this paper proposes a piecewise linearization method specifically designed to be tight, resulting in a reduced problem space a solver can explore faster. We provide numerical results comparing the proposed formulation against two piecewise linearizations from the literature, both from a theoretical point of view and in terms of practical computational performance, with results showing an average speed-up of 2.83 times for our case study. Test cases are carried out on a modified 24-bus IEEE Reliability Test System and a 12-node gas system, considering discrete unit commitment decisions.

We present and analyze a mathematical model to study the feedback between behavior and epidemic spread in a population that is actively assessing and reacting to risk of infection. In our model, a population dynamically forms an opinion that reflects its willingness to engage in risky behavior (e.g., not wearing a mask in a crowded area) or reduce it (e.g., social distancing). We consider SIS epidemic dynamics in which the contact rate within a population adapts as a function of its opinion. For the new coupled model, we prove the existence of two distinct parameter regimes. One regime corresponds to a low baseline infectiousness, and the equilibria of the epidemic spread are identical to those of the standard SIS model. The other regime corresponds to a high baseline infectiousness, and there is a bistability between two new endemic equilibria that reflect an initial preference towards either risk seeking behavior or risk aversion. We prove that risk seeking behavior increases the steady-state infection level in the population compared to the baseline SIS model, whereas risk aversion decreases it. When a population is highly reactive to extreme opinions, we show how risk aversion enables the complete eradication of infection in the population. Extensions of the model to a network of populations or individuals are explored numerically.

Chick sexing, the process of determining the gender of day-old chicks, is a critical task in the poultry industry due to the distinct roles that each gender plays in production. While effective traditional methods achieve high accuracy, color, and wing feather sexing is exclusive to specific breeds, and vent sexing is invasive and requires trained experts. To address these challenges, we propose a novel approach inspired by facial gender classification techniques in humans: facial chick sexing. This new method does not require expert knowledge and aims to reduce training time while enhancing animal welfare by minimizing chick manipulation. We develop a comprehensive system for training and inference that includes data collection, facial and keypoint detection, facial alignment, and classification. We evaluate our model on two sets of images: Cropped Full Face and Cropped Middle Face, both of which maintain essential facial features of the chick for further analysis. Our experiment demonstrates the promising viability, with a final accuracy of 81.89%, of this approach for future practices in chick sexing by making them more universally applicable.

We study the stellar mass function (SMF) and the co-evolution with dark matter halos via abundance matching in the largest redshift range to date $0.25$, we find increased abundances of massive (log$\, M_{\star}/M_{\odot}>10.5$) implying integrated star formation efficiencies (SFE) $\epsilon_{\star}\equiv M_{\star}\, f_{\rm b}^{-1} M_{\rm halo}^{-1} \gtrsim 0.5$. We find a flattening of the SMF at the high-mass end that is better described by a double power law at $z>5.5$. At $z \lesssim 5.5$ it transitions to a Schechter law which coincides with the emergence of the first massive quiescent galaxies in the Universe. We trace the cosmic stellar mass density (SMD) and infer the star formation rate density (SFRD), which at $z>7.5$ agrees remarkably with recent \JWST{} UV luminosity function-derived estimates. However, at $z \lesssim 3.5$, we find significant tension ($\sim 0.3$ dex) with the cosmic star formation (SF) history from instantaneous SF measures, the causes of which remain poorly understood. We infer the stellar-to-halo mass relation (SHMR) and the SFE from abundance matching out to $z=12$, finding a non-monotonic evolution. The SFE has the characteristic strong dependence with mass in the range of $0.02 - 0.2$, and mildly decreases at the low mass end out to $z\sim3.5$. At $z\sim3.5$ the SFE increases sharply from $\sim 0.1$ to approach high SFE of $0.8-1$ by $z\sim 10$ for log$(M_{\rm h}/M_{\odot})\approx11.5$, albeit with large uncertainties. Finally, we use the SHMR to track the SFE and stellar mass growth throughout the halo history and find that they do not grow at the same rate -- from the earliest times up until $z\sim3.5$ the halo growth rate outpaces galaxy assembly, but at $z>3.5$ halo growth stagnates and accumulated gas reservoirs keep the SF going and galaxies outpace halos.

The stellar mass-star formation rate (M$_\star$ - SFR) plane is essential for distinguishing galaxy populations, but how galaxies move within this plane over cosmic time remains unclear. This study aims to describe galaxy migrations in the M$_\star$ - SFR plane by reconstructing star formation histories (SFHs) for a sample of galaxies out to redshift $z < 4$. This provides insights into the physical processes driving star formation. We use data from the COSMOS field, selecting 299131 galaxies at $z < 4$ with COSMOS-Web NIRCam data (m$_\mathrm{F444W} < 27$) over 0.54 deg$^2$. Using the SED modeling code CIGALE with non-parametric SFHs, we derive physical properties and migration vectors for these galaxies. These vectors describe the direction and velocity of evolutionary paths across the M$_\star$ - SFR plane. To assess the accuracy of these vectors, we compare them to results from the Horizon-AGN simulation. Galaxies within the main sequence show low migration amplitudes and dispersed movement directions, indicating oscillation within the main sequence. Most progenitors were already on the main sequence a billion years earlier. Starburst galaxies assembled half their mass in the last 350 Myr and originated from the main sequence. Passive galaxies show uniformly declining SFHs and include massive galaxies already in the passive region at $z = 3.5-4$, with increasing density over time. Using reconstructed SFHs up to $z < 4$, we propose a coherent picture of how galaxies migrate over cosmic time in the M$_\star$ - SFR plane, highlighting the connection between major phases in the star-formation history.

Ensemble methods are known for enhancing the accuracy and robustness of machine learning models by combining multiple base learners. However, standard approaches like greedy or random ensembles often fall short, as they assume a constant weight across samples for the ensemble members. This can limit expressiveness and hinder performance when aggregating the ensemble predictions. In this study, we explore employing neural networks as ensemble methods, emphasizing the significance of dynamic ensembling to leverage diverse model predictions adaptively. Motivated by the risk of learning low-diversity ensembles, we propose regularizing the model by randomly dropping base model predictions during the training. We demonstrate this approach lower bounds the diversity within the ensemble, reducing overfitting and improving generalization capabilities. Our experiments showcase that the dynamic neural ensemblers yield competitive results compared to strong baselines in computer vision, natural language processing, and tabular data., Comment: Preprint under review, 10 pages

We propose a novel method to reconstruct the full posterior distribution of the star formation histories (SFHs) of galaxies from broad-band photometry. Our method combines simulation-based inference (SBI) using a neural network trained with SFHs and photometry from the {\sc Horizon-AGN} hydrodynamical cosmological simulation. We apply it to reconstruct SFHs using COSMOS2020 photometry at redshift $0 9$ had a first event of mass assembly around $z\sim 3$, independent of mass. This work represents a pilot study for the future analysis of the \textit{Euclid} Deep fields that will reach similar depths in alike set of photometric bands, but with over an order-of-magnitude larger area, opening the possibility of deriving SFHs for millions of galaxies in a robust manner.

In this work, we study the function $f_2(G)$ that counts the number of exact factorizations of a finite group $G$. We compute $f_2(G)$ for some well-known families of finite groups and use the results of Wiegold and Williamson \cite{WW} to derive an asymptotic expression for the number of exact factorizations of the alternating group $A_{2^n}$. Finally, we propose several questions about the function $f_2(G)$ that may be of interest for further research., Comment: 19 pages

A proper subdomain $G$ of the unit disk $\mathbb{D}$ is horocyclically convex (horo-convex) if, for every $\omega \in \mathbb{D}\cap \partial G$, there exists a horodisk $H$ such that $\omega \in \partial H$ and $G\cap H=\emptyset$. In this paper we give an internal characterization of these domains, namely, that $G$ is horo-convex if and only if any two points can be joined inside $G$ by a $C^1$ curve composed with finitely many Jordan arcs with hyperbolic curvature in $(-2,2)$. We also give a lower bound for the hyperbolic metric of horo-convex regions and some consequences., Comment: 11 pages

Energy system models are crucial for planning, supporting, and understanding energy transition pathways. Flexible energy modelling tools have emerged to provide practitioners, planners, and decision-makers with various alternatives to represent diverse energy systems, including green hydrogen or exclusively renewable-powered storage assets. The increased interaction between energy sectors, temporal resolution, and extensive geographical scopes have led to large-scale problems posing significant computational challenges. Despite improvements in computing power and linear programming (LP) solvers, large-scale LP models are often simplified, sacrificing fidelity to speed up solutions. This paper aims to debunk the misconception that an LP model's simplicity cannot be improved without sacrificing fidelity. We propose exploiting the graph nature of energy systems using a single building block, the Energy Asset, to reduce computational complexity. By using only one building block, the Energy Asset, we avoid intermediary assets and connections, thus reducing the number of variables by 26% and constraints by 35%. This approach naturally speeds up solving times by 1.27 times without sacrificing model fidelity. Our illustrative case study demonstrates these improvements compared to traditional two-building-block approaches. This paper aims to raise awarenes in the energy modelling community about the quality of LP models and shows that not all LPs are created equal. Our proposed method speeds up energy system models regardless of anticipated advances in software and hardware, allowing for the solution of larger and more detailed models with existing technology., Comment: Preprint submitted to Energy (IJEPES)

In healthcare there is a pursuit for employing interpretable algorithms to assist healthcare professionals in several decision scenarios. Following the Predictive, Descriptive and Relevant (PDR) framework, the definition of interpretable machine learning as a machine-learning model that explicitly and in a simple frame determines relationships either contained in data or learned by the model that are relevant for its functioning and the categorization of models by post-hoc, acquiring interpretability after training, or model-based, being intrinsically embedded in the algorithm design. We overview a selection of eight algorithms, both post-hoc and model-based, that can be used for such purposes., Comment: 26 pages, 5 figures

The dissociation micro-states (DMS) of an $N$-protic acid are described using set theory notation. This facilitates the mathematical description of the dissociation micro-equilibrium (DME). In particular, the DME constants are easily obtained in terms of the dissociation equilibrium constants and the molar fractions of the DMSs. Representing of the DMEs in terms of graph theory allows to identify permutations between DMSs that preserve the vertex-edge connectivity of the graph. These permutations, along with their composition, allow us to identify the direct product of the cyclic group $C_2$, and the symmetric group $S_N$, $C_2\times S_N$, as the graph automorphism group of the micro-dissociation of $N$-protic acids., Comment: 27 pages, 5 figures

We study of the role of galaxy-galaxy interactions and disk instabilities in producing starburst activity in galaxies out to z=4. For this, we use a sample of 387 galaxies with robust total star formation rate measurements from Herschel, gas masses from ALMA, stellar masses and redshifts from multi-band photometry, and JWST/NIRCam rest-frame optical imaging. Using mass-controlled samples, we find an increased fraction of interacting galaxies in the starburst regime at all redshifts out to z=4. This increase correlates with star formation efficiency (SFE), but not with gas fraction. However, the correlation is weak (and only significant out to z=2), which could be explained by the short duration of SFE increase during interaction. In addition, we find that isolated disk galaxies make up a significant fraction of the starburst population. The fraction of such galaxies with star-forming clumps ("clumpy disks") is significantly increased compared to the main-sequence disk population. Furthermore, this fraction directly correlates with SFE. This is direct observational evidence for a long-term increase of SFE maintained due to disk instabilities, contributing to the majority of starburst galaxies in our sample and hence to substantial mass growth in these systems. This result could also be of importance for explaining the growth of the most massive galaxies at z>6., Comment: 14 pages, 5 figures. Submitted to ApJ. Comments welcome

In this work, we explore how resources are distributed among the states generated by quantum random circuits (QRC). We focus on multipartite non-locality, but we also analyze quantum correlations by appealing to different entanglement and non-classicality measures. We compare universal vs non-universal sets of gates to gain insight into the problem of explaining quantum advantage. By comparing the results obtained with ideal (noiseless) vs noisy intermediate-scale quantum (NISQ) devices, we lay the basis of a certification protocol, which aims to quantify how robust is the resources distribution among the states that a given device can generate., Comment: 20 pages, 13 figures

The strict selectivity of the blood-brain barrier (BBB) represents one of the most formidable challenges to successful central nervous system (CNS) drug delivery. Computational methods to generate BBB permeable drugs in silico may be valuable tools in the CNS drug design pipeline. However, in real-world applications, BBB penetration alone is insufficient; rather, after transiting the BBB, molecules must bind to a specific target or receptor in the brain and must also be safe and non-toxic. To discover small molecules that concurrently satisfy these constraints, we use multi-objective generative AI to synthesize drug-like BBB-permeable small molecules. Specifically, we computationally synthesize molecules with predicted binding affinity against dopamine receptor D2, the primary target for many clinically effective antipsychotic drugs. After training several graph neural network-based property predictors, we adapt SyntheMol (Swanson et al., 2024), a recently developed Monte Carlo Tree Search-based algorithm for antibiotic design, to perform a multi-objective guided traversal over an easily synthesizable molecular space. We design a library of 26,581 novel and diverse small molecules containing hits with high predicted BBB permeability and favorable predicted safety and toxicity profiles, and that could readily be synthesized for experimental validation in the wet lab. We also validate top scoring molecules with molecular docking simulation against the D2 receptor and demonstrate predicted binding affinity on par with risperidone, a clinically prescribed D2-targeting antipsychotic. In the future, the SyntheMol-based computational approach described here may enable the discovery of novel neurotherapeutics for currently intractable disorders of the CNS., Comment: 20 pages, 4 figures, Generative and Experimental Perspectives for Biomolecular Design Workshop at the 12th International Conference on Learning Representations

In this paper, we extend the quantum geometric tensor for parameter-dependent curved spaces to higher dimensions, and introduce an equivalent definition that generalizes the Zanardi, et al, formulation of the tensor. The parameter-dependent metric modifies the behavior of both the quantum metric tensor and Berry curvature in a purely geometric way. Our focus is on understanding the distinctions in higher dimensions that emerge when using the generalized tensor compared to the conventional one. Through a comparative analysis, illustrated with examples in two dimensions, we highlight unique quantum geometric properties for both the quantum metric tensor and the Berry curvature. Additionally, we explore differences between analytical and perturbative approaches in solving the problems., Comment: 21 pages

We introduce Chronos, a simple yet effective framework for pretrained probabilistic time series models. Chronos tokenizes time series values using scaling and quantization into a fixed vocabulary and trains existing transformer-based language model architectures on these tokenized time series via the cross-entropy loss. We pretrained Chronos models based on the T5 family (ranging from 20M to 710M parameters) on a large collection of publicly available datasets, complemented by a synthetic dataset that we generated via Gaussian processes to improve generalization. In a comprehensive benchmark consisting of 42 datasets, and comprising both classical local models and deep learning methods, we show that Chronos models: (a) significantly outperform other methods on datasets that were part of the training corpus; and (b) have comparable and occasionally superior zero-shot performance on new datasets, relative to methods that were trained specifically on them. Our results demonstrate that Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks, positioning pretrained models as a viable tool to greatly simplify forecasting pipelines., Comment: Code and model checkpoints available at https://github.com/amazon-science/chronos-forecasting

An important problem in many areas of science is that of recovering interaction networks from simultaneous time-series of many interacting dynamical processes. A common approach is to use the elements of the correlation matrix or its inverse as proxies of the interaction strengths, but the reconstructed networks are necessarily undirected. Transfer entropy methods have been proposed to reconstruct directed networks but the reconstructed network lacks information about interaction strengths. We propose a network reconstruction method that inherits the best of the two approaches by reconstructing a directed weighted network from noisy data under the assumption that the network is sparse and the dynamics are governed by a linear (or weakly-nonlinear) stochastic dynamical system. The two steps of our method are i) constructing an (infinite) family of candidate networks by solving the covariance matrix Lyapunov equation for the state matrix and ii) using L1-regularization to select a sparse solution. We further show how to use prior information on the (non)existence of a few directed edges to drastically improve the quality of the reconstruction.

Faint, star-forming galaxies likely play a dominant role in cosmic reionisation. Strides have been made in recent years to characterise these populations at high redshifts ($z>3$). Now for the first time, with JWST photometry beyond 1$\,\mu m$ in the rest frame, we can derive accurate stellar masses and position these galaxies on the galaxy main sequence. We seek to assess the place of 96 individual Lyman-alpha emitters (LAEs) selected behind the A2744 lensing cluster with MUSE spectroscopy on the galaxy main sequence. We also compare derived stellar masses to Lyman-alpha luminosities and equivalent widths to better quantify the relationship between the Lyman-alpha emission and the host galaxy. These 96 LAEs lie in the redshift range $2.9