Search

JWST has revealed a population of compact `Little Red Dots' (LRDs) at $z\gtrsim4$, with red rest-frame optical and blue UV colors. These objects are likely compact dusty starbursts or heavily reddened AGNs, playing a pivotal role in early black hole growth, dust production, and stellar assembly. We introduce a new photometric selection to identify LRDs over a broad range in redshifts and rest-frame UV-to-NIR colors enabling a more complete census of the population. This method identifies 248 LRDs with F444W$<27$ mag over 263 arcmin$^2$ in the JADES, PRIMER-COSMOS, and UDS fields with MIRI coverage, increasing the number density by $\times$1.7 compared to previous samples, suggesting that previous census were underestimated. Most LRDs are detected in MIRI/F770W but only 7% (17) are detected in F1800W. We use MIRI-based rest-frame [1$-$3 $\mu$m] colors to trace dust emission. F1800W-detected LRDs have a median [1$-$3 $\mu$m]$=1.5$ mag, with a broad scatter indicative of diverse dust emission properties. About 20% exhibit [1$-$3 $\mu$m]$<1$ mag colors consistent with negligible dust emission, but the majority show significant dust emission at 3 $\mu$m (f$^{\rm dust}_{3\mu m}\lesssim0.8$) from the galaxy ISM or a hot-dust-deficient AGN torus. A correlation between bluer UV-to-NIR colors and stronger IR emission suggests that the bluest LRDs may resemble unobscured QSOs. We report a LRD at $z_{\rm spec}=3.1386$, detected in MIRI, Spitzer/MIPS, and Herschel/PACS. Its IR SED rises steeply at $\lambda_{\rm rest}>6~\mu$m and peaks near $\sim40~\mu$m, providing the first direct evidence of warm dust emission (T$=50-100$ K) in a LRD., Comment: The new photometric selection is shown in Figure 1. The mid-IR detected LRD is in Figure 11

Effective monitoring of wildlife is critical for assessing biodiversity and ecosystem health, as declines in key species often signal significant environmental changes. Birds, particularly ground-nesting species, serve as important ecological indicators due to their sensitivity to environmental pressures. Camera traps have become indispensable tools for monitoring nesting bird populations, enabling data collection across diverse habitats. However, the manual processing and analysis of such data are resource-intensive, often delaying the delivery of actionable conservation insights. This study presents an AI-driven approach for real-time species detection, focusing on the curlew (Numenius arquata), a ground-nesting bird experiencing significant population declines. A custom-trained YOLOv10 model was developed to detect and classify curlews and their chicks using 3/4G-enabled cameras linked to the Conservation AI platform. The system processes camera trap data in real-time, significantly enhancing monitoring efficiency. Across 11 nesting sites in Wales, the model achieved high performance, with a sensitivity of 90.56%, specificity of 100%, and F1-score of 95.05% for curlew detections, and a sensitivity of 92.35%, specificity of 100%, and F1-score of 96.03% for curlew chick detections. These results demonstrate the capability of AI-driven monitoring systems to deliver accurate, timely data for biodiversity assessments, facilitating early conservation interventions and advancing the use of technology in ecological research.

Camera traps offer enormous new opportunities in ecological studies, but current automated image analysis methods often lack the contextual richness needed to support impactful conservation outcomes. Here we present an integrated approach that combines deep learning-based vision and language models to improve ecological reporting using data from camera traps. We introduce a two-stage system: YOLOv10-X to localise and classify species (mammals and birds) within images, and a Phi-3.5-vision-instruct model to read YOLOv10-X binding box labels to identify species, overcoming its limitation with hard to classify objects in images. Additionally, Phi-3.5 detects broader variables, such as vegetation type, and time of day, providing rich ecological and environmental context to YOLO's species detection output. When combined, this output is processed by the model's natural language system to answer complex queries, and retrieval-augmented generation (RAG) is employed to enrich responses with external information, like species weight and IUCN status (information that cannot be obtained through direct visual analysis). This information is used to automatically generate structured reports, providing biodiversity stakeholders with deeper insights into, for example, species abundance, distribution, animal behaviour, and habitat selection. Our approach delivers contextually rich narratives that aid in wildlife management decisions. By providing contextually rich insights, our approach not only reduces manual effort but also supports timely decision-making in conservation, potentially shifting efforts from reactive to proactive management., Comment: 32 Pages, 22 images

Large Language Models (LLMs) and Vision Language Models (VLMs) possess extensive knowledge and exhibit promising reasoning abilities; however, they still struggle to perform well in complex, dynamic environments. Real-world tasks require handling intricate interactions, advanced spatial reasoning, long-term planning, and continuous exploration of new strategies-areas in which we lack effective methodologies for comprehensively evaluating these capabilities. To address this gap, we introduce BALROG, a novel benchmark designed to assess the agentic capabilities of LLMs and VLMs through a diverse set of challenging games. Our benchmark incorporates a range of existing reinforcement learning environments with varying levels of difficulty, including tasks that are solvable by non-expert humans in seconds to extremely challenging ones that may take years to master (e.g., the NetHack Learning Environment). We devise fine-grained metrics to measure performance and conduct an extensive evaluation of several popular open-source and closed-source LLMs and VLMs. Our findings indicate that while current models achieve partial success in the easier games, they struggle significantly with more challenging tasks. Notably, we observe severe deficiencies in vision-based decision-making, as models perform worse when visual representations of the environments are provided. We release BALROG as an open and user-friendly benchmark to facilitate future research and development in the agentic community., Comment: Preprint, under review

JWST has revealed a large population of compact, red galaxies at $z>4$ known as Little Red Dots (LRDs). We analyze the spectral energy distributions (SEDs) of 95 LRDs from the JWST PRIMER survey with complete photometric coverage from $1-18\ \mu$m using NIRCam and MIRI imaging, representing the most extensive SED analysis on a large LRD sample with long-wavelength MIRI data. We examine SED models in which either galaxy or active galactic nucleus (AGN) emission dominates the rest-frame UV or optical continuum, extracting physical properties to explore each scenario's implications. In the galaxy-only model, we find massive, dusty stellar populations alongside unobscured, low-mass components, hinting at inhomogeneous obscuration. The AGN-only model indicates dusty, luminous AGNs with low hot dust fractions compared to typical quasars. A hybrid AGN and galaxy model suggests low-mass, unobscured galaxies in the UV, with stellar mass estimates spanning $\sim$2 dex across the different models, underscoring the need for caution in interpreting LRD stellar masses. With MIRI photometry, the galaxy-only model produces stellar masses within cosmological limits, but extremely high stellar mass densities are inferred. The hybrid model infers highly overmassive black holes exceeding those in recently reported high-redshift AGNs, hinting at a partial AGN contribution to the rest-optical continuum or widespread super-Eddington accretion. Our findings highlight the extreme conditions required for both AGN or galaxy dominated scenarios in LRDs, supporting a mixed contribution to the red continuum, or novel scenarios to explain the observed emission., Comment: 22 pages, 10 figures, submitted to ApJ. Machine-readable form of Table 2 available at: https://github.com/geneckleung/lrd_primer_miri

Purpose: To present a novel generalized MR image reconstruction based on pseudoinversion of the encoding matrix (Pinv-Recon) as a simple yet powerful method, and demonstrate its computational feasibility for diverse MR imaging applications. Methods: MR image encoding constitutes a linear mapping of the unknown image to the measured k-space data mediated via an encoding matrix ($ data = Encode \times image$). Pinv-Recon addresses MR image reconstruction as a linear inverse problem ($image = Encode^{-1} \times data$), explicitly calculating the Moore-Penrose pseudoinverse of the encoding matrix using truncated singular value decomposition (tSVD). Using a discretized, algebraic notation, we demonstrate constructing a generalized encoding matrix by stacking relevant encoding mechanisms (e.g., gradient encoding, coil sensitivity encoding, chemical shift inversion) and encoding distortions (e.g., off-center positioning, B$_0$ inhomogeneity, spatiotemporal gradient imperfections, transient relaxation effects). Iterative reconstructions using the explicit generalized encoding matrix, and the computation of the spatial-response-function (SRF) and noise amplification, were demonstrated. Results: We evaluated the computation times and memory requirements (time ~ (size of the encoding matrix)$^{1.4}$). Using the Shepp-Logan phantom, we demonstrated the versatility of the method for various intertwined MR image encoding and distortion mechanisms, achieving better MSE, PSNR and SSIM metrics than conventional methods. A diversity of datasets, including the ISMRM CG-SENSE challenge, were used to validate Pinv-Recon. Conclusion: Although pseudo-inversion of large encoding matrices was once deemed computationally intractable, recent advances make Pinv-Recon feasible. It has great promise for both research and clinical applications, and for educational use., Comment: 26 pages, 8 figures, 2 tables (+ Supplementary Material). Submitted to Magnetic Resonance in Medicine

Software engineers mainly write code by editing existing programs. In contrast, large language models (LLMs) autoregressively synthesize programs in a single pass. One explanation for this is the scarcity of open-sourced edit data. While high-quality instruction data for code synthesis is already scarce, high-quality edit data is even scarcer. To fill this gap, we develop a synthetic data generation algorithm called LintSeq. This algorithm refactors existing code into a sequence of code edits by using a linter to procedurally sample across the error-free insertions that can be used to sequentially write programs. It outputs edit sequences as text strings consisting of consecutive program diffs. To test LintSeq, we use it to refactor a dataset of instruction + program pairs into instruction + program-diff-sequence tuples. Then, we instruction finetune a series of smaller LLMs ranging from 2.6B to 14B parameters on both the re-factored and original versions of this dataset, comparing zero-shot performance on code synthesis benchmarks. We show that during repeated sampling, edit sequence finetuned models produce more diverse programs than baselines. This results in better inference-time scaling for benchmark coverage as a function of samples, i.e. the fraction of problems "pass@k" solved by any attempt given "k" tries. For example, on HumanEval pass@50, small LLMs finetuned on synthetic edit sequences are competitive with GPT-4 and outperform models finetuned on the baseline dataset by +20% (+/-3%) in absolute score. Finally, we also pretrain our own tiny LMs for code understanding. We show that finetuning tiny models on synthetic code edits results in state-of-the-art code synthesis for the on-device model class. Our 150M parameter edit sequence LM matches or outperforms code models with twice as many parameters, both with and without repeated sampling, including Codex and AlphaCode.

We present the properties of a massive, large, dusty, metal-rich, star-forming galaxy at z_spec=6.73. GOODSN-100182 was observed with JWST/NIRSpec as part of the AURORA survey, and is also covered by public multi-wavelength HST and JWST imaging. While the large mass of GOODSN-100182 (~10^10 M_sun) was indicated prior to JWST, NIRCam rest-optical imaging now reveals the presence of an extended disk (r_eff~1.5 kpc). In addition, the NIRSpec R~1000 spectrum of GOODSN-100182 includes the detection of a large suite of rest-optical nebular emission lines ranging in wavelength from [OII]3727 up to [NII]6583. The ratios of Balmer lines suggest significant dust attenuation (E(B-V)_gas=0.40+0.10/-0.09), consistent with the red rest-UV slope inferred for GOODSN-100182 (beta=-0.50+/-0.09). The star-formation rate based on dust-corrected H-alpha emission is log(SFR(H-alpha)/ M_sun/yr)=2.02+0.13/-0.14, well above the z~7 star-forming main sequence in terms of specific SFR. Strikingly, the ratio of [NII]6583/H-alpha emission suggests almost solar metallicity, as does the ratio ([OIII]5007/H-beta)/([NII]6583/H-alpha) and the detection of the faint [FeII]4360 emission feature, whereas the [OIII]5007/[OII]3727 ratio suggests roughly 50% solar metallicity. Overall, the excitation and ionization properties of GOODSN-100182 more closely resemble those of typical star-forming galaxies at z~2-3 rather than z~7. Based on public spectroscopy of the GOODS-N field, we find that GOODSN-100182 resides within a significant galaxy overdensity, and is accompanied by a spectroscopically-confirmed neighbor galaxy. GOODSN-100182 demonstrates the existence of mature, chemically-enriched galaxies within the first billion years of cosmic time, whose properties must be explained by galaxy formation models., Comment: 16 pages, 13 figures, submitted to ApJ

We analyze JWST MIRI/MRS IFU observations of three Seyferts and showcase the intriguing polycyclic aromatic hydrocarbon (PAH) emission characteristics in regions of $\sim 500\,\rm pc$ scales over or around their active galactic nuclei (AGN). Combining the model predictions and the measurements of PAH features and other infrared emission lines, we find that the central regions containing a high fraction of neutral PAHs with small sizes, e.g., those in ESO137-G034, are in highly heated environments, due to collisional shock heating, with hard and moderately intense radiation fields. Such environments are proposed to be associated with inhibited growth or preferential erosion of PAHs, decreasing the average PAH size and the overall abundance of PAHs. We additionally find that the central regions containing a high fraction of ionized PAHs with large sizes, e.g., those in MCG-05-23-016, are likely experiencing severe photo-ionization because of the radiative effects from the radiative shock precursor besides the AGN. The severe photo-ionization can contribute to the ionization of all PAHs and further destruction of small PAHs. Overall, different Seyferts, even different regions in the same galaxy, e.g., those in NGC\,3081, can contain PAH populations of different properties. Specifically, Seyferts that exhibit similar PAH characteristics to ESO137-G034 and MCG-05-23-016 also tend to have similar emission line properties to them, suggesting that the explanations for PAH characteristics of ESO137-G034 and MCG-05-23-016 may also apply generally. These results have promising application in the era of JWST, especially in diagnosing different (i.e., radiative, and kinetic) AGN feedback modes., Comment: ApJL accepted on September 26th, title slightly modified in accordance with ApJL standards, Fig. 2 updated with additional labels

Computational design of protein-binding proteins is a fundamental capability with broad utility in biomedical research and biotechnology. Recent methods have made strides against some target proteins, but on-demand creation of high-affinity binders without multiple rounds of experimental testing remains an unsolved challenge. This technical report introduces AlphaProteo, a family of machine learning models for protein design, and details its performance on the de novo binder design problem. With AlphaProteo, we achieve 3- to 300-fold better binding affinities and higher experimental success rates than the best existing methods on seven target proteins. Our results suggest that AlphaProteo can generate binders "ready-to-use" for many research applications using only one round of medium-throughput screening and no further optimization., Comment: 45 pages, 17 figures

In dense retrieval, deep encoders provide embeddings for both inputs and targets, and the softmax function is used to parameterize a distribution over a large number of candidate targets (e.g., textual passages for information retrieval). Significant challenges arise in training such encoders in the increasingly prevalent scenario of (1) a large number of targets, (2) a computationally expensive target encoder model, (3) cached target embeddings that are out-of-date due to ongoing training of target encoder parameters. This paper presents a simple and highly scalable response to these challenges by training a small parametric corrector network that adjusts stale cached target embeddings, enabling an accurate softmax approximation and thereby sampling of up-to-date high scoring "hard negatives." We theoretically investigate the generalization properties of our proposed target corrector, relating the complexity of the network, staleness of cached representations, and the amount of training data. We present experimental results on large benchmark dense retrieval datasets as well as on QA with retrieval augmented language models. Our approach matches state-of-the-art results even when no target embedding updates are made during training beyond an initial cache from the unsupervised pre-trained model, providing a 4-80x reduction in re-embedding computational cost., Comment: ICML 2024

The rapid decline in global biodiversity demands innovative conservation strategies. This paper examines the use of artificial intelligence (AI) in wildlife conservation, focusing on the Conservation AI platform. Leveraging machine learning and computer vision, Conservation AI detects and classifies animals, humans, and poaching-related objects using visual spectrum and thermal infrared cameras. The platform processes this data with convolutional neural networks (CNNs) and Transformer architectures to monitor species, including those which are critically endangered. Real-time detection provides the immediate responses required for time-critical situations (e.g. poaching), while non-real-time analysis supports long-term wildlife monitoring and habitat health assessment. Case studies from Europe, North America, Africa, and Southeast Asia highlight the platform's success in species identification, biodiversity monitoring, and poaching prevention. The paper also discusses challenges related to data quality, model accuracy, and logistical constraints, while outlining future directions involving technological advancements, expansion into new geographical regions, and deeper collaboration with local communities and policymakers. Conservation AI represents a significant step forward in addressing the urgent challenges of wildlife conservation, offering a scalable and adaptable solution that can be implemented globally., Comment: 13 pages, 13 figures

We study the 3.4-4.4$\mu$m fundamental rovibrational band of H3+, a key tracer of the ionization of the molecular interstellar medium (ISM), in a sample of 12 local (d< 400 Mpc) ultra/luminous infrared galaxies (U/LIRGs) observed with JWST/NIRSpec. The P, Q, and R branches of the band are detected in 13 out of 20 analyzed regions within these U/LIRGs, which increases the number of extragalactic H3+ detections by a factor of 6. For the first time in the ISM, the H3+ band is observed in emission in 3 of these regions. In the remaining 10 regions, the band is seen in absorption. The absorptions are produced toward the 3.4-4.4$\mu$m hot dust continuum rather than toward the stellar continuum, indicating that they likely originate in clouds associated with the dust continuum source. The H3+ band is undetected in Seyfert-like U/LIRGs where the mildly obscured X-ray radiation from the AGN might limit the abundance of this molecule. For the detections, the H3+ abundances, N(H3+)/N_H = (0.5-5.5)x10^-7, imply relatively high ionization rates between 3x10^-16 and >4x10^-15 s^-1, which are likely associated with high-energy cosmic rays. In half of the targets the absorptions are blue-shifted by 50-180 km/s, which are lower than the molecular outflow velocities measured using other tracers such as OH 119$\mu$m or rotational CO lines. This suggests that H3+ traces gas close to the outflow launching sites before it has been fully accelerated. We used nonlocal thermodynamic equilibrium models to investigate the physical conditions of these clouds. In 7 out of 10 objects, the H3+ excitation is consistent with inelastic collisions with H2 in warm translucent molecular clouds (T_kin ~ 250-500 K and n(H2) ~ 10^(2-3) cm^-3). In three objects, dominant infrared pumping excitation is required to explain the absorptions from the (3,0) and (2,1) levels of H3+ detected for the first time in the ISM., Comment: Accepted for publication in A&A Letters. 6 pages, 5 figures. Appendix: 5 pages, 4 figures

JWST has enabled the detection of the UV continuum of galaxies at z>10, evidencing a population of extremely blue, potentially dust-free galaxies. Interpreting the UV spectra of galaxies as they redden is complicated by the well-known degeneracy between stellar ages, dust, and nebular continuum. The main goal of this paper is to develop a theoretical model for the relationship between galaxy UV slopes, bursty star formation histories, dust evolution, and the contribution from nebular regions. We accomplish this via cosmological zoom-in simulations, and in specific, build a layered model where we simulate the UV slopes of galaxies with increasingly complex physics. Our main results follow. (i) Unattenuated stellar populations with no nebular emission exhibit a diverse range of intrinsic UV slopes, with values ranging from beta ~ -3 --> -2.2 due to long delays between bursts. This is manifested by an inverse correlation between the intrinsic UV slope and sSFR for early galaxies such that higher sSFR corresponds to bluer UV slopes. (ii) When including dust, our model galaxies demonstrate a rapid rise in dust obscuration between z ~ 8-10. This increase in dust mass is due to high grain-grain shattering rates, and enhanced growth per unit dust mass in very small grains, resulting in UV-detected galaxies at z ~ 12 descending into ALMA-detectable galaxies by z ~ 6. The rapid rise in dust content at z ~ 8-10 leads to a systematic reddening of the UV slopes during this redshift range. (iii) The inclusion of nebular continuum reddens the UV slope by a median factor Delta beta ~ 0.2-0.4. However, when including nebular continuum, our highest redshift galaxies (z~12) are insufficiently blue compared to observations; this may imply an evolving escape fraction from HII regions with redshift., Comment: Submitted to ApJ; comments welcome

In the face of difficult exploration problems in reinforcement learning, we study whether giving an agent an object-centric mapping (describing a set of items and their attributes) allow for more efficient learning. We found this problem is best solved hierarchically by modelling items at a higher level of state abstraction to pixels, and attribute change at a higher level of temporal abstraction to primitive actions. This abstraction simplifies the transition dynamic by making specific future states easier to predict. We make use of this to propose a fully model-based algorithm that learns a discriminative world model, plans to explore efficiently with only a count-based intrinsic reward, and can subsequently plan to reach any discovered (abstract) states. We demonstrate the model's ability to (i) efficiently solve single tasks, (ii) transfer zero-shot and few-shot across item types and environments, and (iii) plan across long horizons. Across a suite of 2D crafting and MiniHack environments, we empirically show our model significantly out-performs state-of-the-art low-level methods (without abstraction), as well as performant model-free and model-based methods using the same abstraction. Finally, we show how to learn low level object-perturbing policies via reinforcement learning, and the object mapping itself by supervised learning., Comment: Preprint. Additional results

We use JWST/NIRSpec observations from the Assembly of Ultradeep Rest-optical Observations Revealing Astrophysics (AURORA) survey to constrain the shape of the nebular attenuation curve of a star-forming galaxy at z=4.41, GOODSN-17940. We utilize 11 unblended HI recombination lines to derive the attenuation curve spanning optical to near-infrared wavelengths (3751-9550 \r{A}). We then leverage a high-S/N spectroscopic detection of the rest-frame ultraviolet continuum in combination with rest-UV photometric measurements to constrain the shape of the curve at ultraviolet wavelengths. While this UV constraint is predominantly based on stellar emission, the large measured equivalent widths of H$\alpha$ and H$\beta$ indicate that GOODSN-17940 is dominated by an extremely young stellar population <10 Myr in age such that the UV stellar continuum experiences the same attenuation as the nebular emission. The resulting combined nebular attenuation curve spans 1400-9550 \r{A} and has a shape that deviates significantly from commonly assumed dust curves in high-redshift studies. Relative to the Milky Way, SMC, and Calzetti curves, the new curve has a steeper slope at long wavelengths ($\lambda>5000$ \r{A}) while displaying a similar slope across blue-optical wavelengths ($\lambda=3750-5000$ \r{A}). In the ultraviolet, the new curve is shallower than the SMC and Calzetti curves and displays no significant 2175 \r{A} bump. This work demonstrates that the most commonly assumed dust curves are not appropriate for all high-redshift galaxies. These results highlight the ability to derive nebular attenuation curves for individual high-redshift sources with deep JWST/NIRSpec spectroscopy, thereby improving the accuracy of physical properties inferred from nebular emission lines., Comment: 21 pages, 6 figures, 1 table, submitted to ApJ

Our hypothesis is that UDA using diffusion-weighted images, generated with a unified model, offers a promising and reliable strategy for enhancing the performance of supervised learning models in multi-site prostate lesion detection, especially when various b-values are present. This retrospective study included data from 5,150 patients (14,191 samples) collected across nine different imaging centers. A novel UDA method using a unified generative model was developed for multi-site PCa detection. This method translates diffusion-weighted imaging (DWI) acquisitions, including apparent diffusion coefficient (ADC) and individual DW images acquired using various b-values, to align with the style of images acquired using b-values recommended by Prostate Imaging Reporting and Data System (PI-RADS) guidelines. The generated ADC and DW images replace the original images for PCa detection. An independent set of 1,692 test cases (2,393 samples) was used for evaluation. The area under the receiver operating characteristic curve (AUC) was used as the primary metric, and statistical analysis was performed via bootstrapping. For all test cases, the AUC values for baseline SL and UDA methods were 0.73 and 0.79 (p<.001), respectively, for PI-RADS>=3, and 0.77 and 0.80 (p<.001) for PI-RADS>=4 PCa lesions. In the 361 test cases under the most unfavorable image acquisition setting, the AUC values for baseline SL and UDA were 0.49 and 0.76 (p<.001) for PI-RADS>=3, and 0.50 and 0.77 (p<.001) for PI-RADS>=4 PCa lesions. The results indicate the proposed UDA with generated images improved the performance of SL methods in multi-site PCa lesion detection across datasets with various b values, especially for images acquired with significant deviations from the PI-RADS recommended DWI protocol (e.g. with an extremely high b-value)., Comment: Accept at Radiology: Artificial Intelligence. Journal reference and external DOI will be added once published

Mechanistic compartmental models are widely used in epidemiology to study the dynamics of infectious disease transmission. These models have significantly contributed to designing and evaluating effective control strategies during pandemics. However, the increasing complexity and the number of parameters needed to describe rapidly evolving transmission scenarios present significant challenges for parameter estimation due to intractable likelihoods. To overcome this issue, likelihood-free methods have proven effective for accurately and efficiently fitting these models to data. In this study, we focus on approximate Bayesian computation (ABC) and synthetic likelihood methods for parameter inference. We develop a method that employs ABC to select the most informative subset of summary statistics, which are then used to construct a synthetic likelihood for posterior sampling. Posterior sampling is performed using Hamiltonian Monte Carlo as implemented in the Stan software. The proposed algorithm is demonstrated through simulation studies, showing promising results for inference in a simulated epidemic scenario.

The application of machine learning in medicine and healthcare has led to the creation of numerous diagnostic and prognostic models. However, despite their success, current approaches generally issue predictions using data from a single modality. This stands in stark contrast with clinician decision-making which employs diverse information from multiple sources. While several multimodal machine learning approaches exist, significant challenges in developing multimodal systems remain that are hindering clinical adoption. In this paper, we introduce a multimodal framework, AutoPrognosis-M, that enables the integration of structured clinical (tabular) data and medical imaging using automated machine learning. AutoPrognosis-M incorporates 17 imaging models, including convolutional neural networks and vision transformers, and three distinct multimodal fusion strategies. In an illustrative application using a multimodal skin lesion dataset, we highlight the importance of multimodal machine learning and the power of combining multiple fusion strategies using ensemble learning. We have open-sourced our framework as a tool for the community and hope it will accelerate the uptake of multimodal machine learning in healthcare and spur further innovation.

We present a relativistic disk reflection model based on the geometry calculated using analytical formulae for super-Eddington accretion flows. This model features a slim disk geometry where the inner disk thickness is proportional to radius, becoming thicker as the mass accretion rate increases. The slim disk profile reduces the brightness of the blue horn in the Fe K emission line for a fixed emissivity and significantly changes the intensity profile for a lamppost geometry. The model is constructed assuming a spherically symmetric spacetime. It can be used for any kind of sources showing fluorescent reflection features and predicted to have slim accretion disks, like slow rotating black holes in X-ray binaries, active galactic nuclei, tidal disruption events, and neutron star X-ray binaries. To show the capability of the model, we use the 2017 \textit{NICER} and \textit{NuSTAR} data of the ultraluminous X-ray transient Swift~J0243.6+6124., Comment: 9 pages, 4 figures

Galactic stellar haloes are largely composed of the remnants of galaxies accreted during the assembly of their host galaxies, and hence their properties reflect the mass spectrum and post-accretion evolution of their satellites. As the nature of dark matter (DM) can affect both, we explore how the properties of the accreted stellar component vary across cold (CDM), warm (WDM) and self-interacting (SIDM) models. We do this by studying accreted stellar populations around eight MW-mass haloes using cosmological hydrodynamical simulations based on the EAGLE galaxy formation model, in which we find that the accreted stellar mass remains similar across models. Contrary to WDM, which only presents minor differences relative to CDM, the distribution of accreted stars in SIDM changes significantly within $0.05R_{200}$ ($10\,\mathrm{kpc}$). The central density reduces to $\langle \rho^{\mathrm{SIDM}}_{\mathrm{exsitu}} / \rho^{\mathrm{CDM}}_{\mathrm{exsitu}} \rangle = 0.3$ and has a shallower radial dependence, with logarithmic density slopes of $\langle \alpha_{\mathrm{SIDM}} \rangle = -1.4$ vs $\langle \alpha_{\mathrm{CDM}} \rangle = -1.7$. Additionally, stars are on more tangential orbits than their CDM counterparts, with a change in the velocity anisotropy of $\langle \Delta \beta \rangle = - 0.2$. Finally, SIDM stellar haloes have the largest number and prominence of overdensities in radius vs radial velocity space. This is due to a combination of shorter stellar halo progenitor merging timescales and shallower host potentials, with the former resulting in less time for dynamical friction and radialisation to operate. In summary, we show that the phase-space structure of Galactic stellar haloes encode key information that can be used to distinguish and rule out different DM models., Comment: 13 pages, 8 figures. Submitted to MNRAS

We present results on the emission-line properties of z=1.4-7.5 star-forming galaxies in the Assembly of Ultradeep Rest-optical Observations Revealing Astrophysics (AURORA) Cycle 1 JWST/NIRSpec program. Based on its depth, continuous wavelength coverage from 1--5 microns, and medium spectral resolution (R~1000), AURORA includes detections of a large suite of nebular emission lines spanning a broad range in rest wavelength. We investigate the locations of AURORA galaxies in multiple different emission-line diagrams, including traditional "BPT" diagrams of [OIII]/Hbeta vs. [NII]/Halpha, [SII]/Halpha, and [OI]/Halpha, and the "ionization-metallicity" diagram of [OIII]/[OII] (O32) vs. ([OIII]+[OII])/Hbeta (R23). We also consider a bluer rest-frame "ionization-metallicity" diagram introduced recently to characterize z>10 galaxies: [NeIII]/[OII] vs. ([NeIII]+[OII])/Hdelta; as well as longer-wavelength diagnostic diagrams extending into the rest-frame near-IR: [OIII]/Hbeta vs. [SIII]/[SII] (S32); and HeI/Pagamma and [SIII]/Pagamma vs. [FeII]/Pabeta. With a significant boost in signal-to-noise and large, representative samples of individual galaxy detections, the AURORA emission-line diagrams presented here definitively confirm a physical picture in which chemically-young, alpha-enhanced, massive stars photoionize the ISM in distant galaxies with a harder ionizing spectrum at fixed nebular metallicity than in their z~0 counterparts. We also uncover previously unseen evolution prior to z~2 in the [OIII]/Hbeta vs. [NII]/Halpha diagram, which motivates deep NIRSpec observations at even higher redshift. Finally, we present the first statistical sample of rest-frame near-IR emission-line diagnostics in star-forming galaxies at high redshift. In order to truly interpret rest-frame near-IR line ratios including [FeII], we must obtain better constraints on dust depletion in the high-redshift ISM., Comment: 17 pages, 8 figures, submitted to ApJ

We present the number densities and physical properties of the bright galaxies spectroscopically confirmed at $z\sim7-14$. Our sample is composed of 60 galaxies at $z_\mathrm{spec}\sim7-14$, including recently-confirmed galaxies at $z_\mathrm{spec}=12.34-14.32$ with JWST, as well as new confirmations at $z_\mathrm{spec}=6.583-7.643$ with $-24< M_\mathrm{UV}< -21$ mag using ALMA and Keck. Our JWST/NIRSpec observations have also revealed that very bright galaxy candidates at $z\sim10-13$ identified from ground-based telescope images before JWST are passive galaxies at $z\sim3-4$, emphasizing the necessity of strict screening and spectroscopy in the selection of the brightest galaxies at $z>10$. The UV luminosity functions derived from these spectroscopic results are consistent with a double power-law function, showing tensions with theoretical models at the bright end. To understand the origin of the overabundance of bright galaxies, we investigate their morphologies using JWST/NIRCam high-resolution images obtained in various surveys including PRIMER and COSMOS-Web. We find that $\sim70\%$ of the bright galaxies at $z\sim7$ exhibit clumpy morphologies with multiple sub-components, suggesting merger-induced starburst activity, which is consistent with SED fitting results showing bursty star formation histories. At $z\gtrsim10$, bright galaxies are classified into two types of galaxies; extended ones with weak high-ionization emission lines, and compact ones with strong high-ionization lines including NIV]$\lambda$1486, indicating that at least two different processes (e.g., merger-induced starburst and compact star formation/AGN) are shaping the physical properties of the brightest galaxies at $z\gtrsim10$ and are responsible for their overabundance., Comment: 34 pages, 22 figures, Accepted for Publication in ApJ

We introduce Cambrian-1, a family of multimodal LLMs (MLLMs) designed with a vision-centric approach. While stronger language models can enhance multimodal capabilities, the design choices for vision components are often insufficiently explored and disconnected from visual representation learning research. This gap hinders accurate sensory grounding in real-world scenarios. Our study uses LLMs and visual instruction tuning as an interface to evaluate various visual representations, offering new insights into different models and architectures -- self-supervised, strongly supervised, or combinations thereof -- based on experiments with over 20 vision encoders. We critically examine existing MLLM benchmarks, address the difficulties involved in consolidating and interpreting results from various tasks, and introduce a new vision-centric benchmark, CV-Bench. To further improve visual grounding, we propose the Spatial Vision Aggregator (SVA), a dynamic and spatially-aware connector that integrates high-resolution vision features with LLMs while reducing the number of tokens. Additionally, we discuss the curation of high-quality visual instruction-tuning data from publicly available sources, emphasizing the importance of data source balancing and distribution ratio. Collectively, Cambrian-1 not only achieves state-of-the-art performance but also serves as a comprehensive, open cookbook for instruction-tuned MLLMs. We provide model weights, code, supporting tools, datasets, and detailed instruction-tuning and evaluation recipes. We hope our release will inspire and accelerate advancements in multimodal systems and visual representation learning., Comment: NeurIPS 2024 (Oral). Website at https://cambrian-mllm.github.io

Colloids can acquire a dipolar interaction in the presence of an external electric field. At high field strength, the particles form strings in the field direction. However at weaker field strength, competition with isotropic interactions is expected. One means to investigate this interplay between dipolar and isotropic interactions is to consider clusters of such particles. We have therefore identified, using the GMIN basinhopping tool, a rich library of lowest energy clusters of a dipolar colloidal system where the dipole orientation is fixed with respect to the z-axis, and the dipole strength is varied for m--membered clusters of 7 <= m <=13. In the regime where the isotropic and dipolar interactions are comparable, we find elongated polytetrahedral, octahedral and spiral clusters as well as a set of non--rigid clusters which emerge close to the transition to strings. We further implement a search algorithm which identifies these minimum energy clusters in bulk systems using the topological cluster classification [J. Chem. Phys. 139 234506 (2013)]. We demonstrate this methodology with computer simulations which shows instances of these clusters as a function of dipole strength., Comment: 11 pages

Long COVID is characterized by persistent symptoms, particularly pulmonary impairment, which necessitates advanced imaging for accurate diagnosis. Hyperpolarised Xenon-129 MRI (XeMRI) offers a promising avenue by visualising lung ventilation, perfusion, as well as gas transfer. Integrating functional data from XeMRI with structural data from Computed Tomography (CT) is crucial for comprehensive analysis and effective treatment strategies in long COVID, requiring precise data alignment from those complementary imaging modalities. To this end, CT-MRI registration is an essential intermediate step, given the significant challenges posed by the direct alignment of CT and Xe-MRI. Therefore, we proposed an end-to-end multimodal deformable image registration method that achieves superior performance for aligning long-COVID lung CT and proton density MRI (pMRI) data. Moreover, our method incorporates a novel Multi-perspective Loss (MPL) function, enhancing state-of-the-art deep learning methods for monomodal registration by making them adaptable for multimodal tasks. The registration results achieve a Dice coefficient score of 0.913, indicating a substantial improvement over the state-of-the-art multimodal image registration techniques. Since the XeMRI and pMRI images are acquired in the same sessions and can be roughly aligned, our results facilitate subsequent registration between XeMRI and CT, thereby potentially enhancing clinical decision-making for long COVID management.

Pseudo-labeling is a popular semi-supervised learning technique to leverage unlabeled data when labeled samples are scarce. The generation and selection of pseudo-labels heavily rely on labeled data. Existing approaches implicitly assume that the labeled data is gold standard and 'perfect'. However, this can be violated in reality with issues such as mislabeling or ambiguity. We address this overlooked aspect and show the importance of investigating labeled data quality to improve any pseudo-labeling method. Specifically, we introduce a novel data characterization and selection framework called DIPS to extend pseudo-labeling. We select useful labeled and pseudo-labeled samples via analysis of learning dynamics. We demonstrate the applicability and impact of DIPS for various pseudo-labeling methods across an extensive range of real-world tabular and image datasets. Additionally, DIPS improves data efficiency and reduces the performance distinctions between different pseudo-labelers. Overall, we highlight the significant benefits of a data-centric rethinking of pseudo-labeling in real-world settings., Comment: Published in the Journal of Data-centric Machine Learning Research (DMLR) *Seedat & Huynh contributed equally

We present a systematic study of mid-infrared spectra of galaxies including star-forming galaxies and Active Galactic Nuclei observed with JWST MIRI-MRS and NIRSpec-IFU. We focus on the relative variations of the 3.3, 6.2, 7.7, 11.3, 12.7 and 17 micron Polycyclic Aromatic Hydrocarbon (PAH) features within spatially resolved regions of galaxies including NGC 3256, NGC 7469, VV 114, II Zw96 and NGC 5728. Using theoretical PAH models and extending our earlier work, we introduce a new PAH diagnostic involving the 17 micron PAH feature. To determine the drivers of PAH band variations in galaxies, we compare observed PAH spectral bands to predictions from theoretical PAH models. We consider extinction, dehydrogenation and PAH size and charge as possible drivers of PAH band variations. We find a surprising uniformity in PAH size distribution among the spatially resolved regions of the galaxies studied here, with no evidence for preferential destruction of the smallest grains, contrary to earlier findings. Neither extinction nor dehydrogenation play a crucial role in setting the observed PAH bands. Instead, we find that PAH charge plays a significant role in PAH inter-band variations. We find a tight relation between PAH charge and the intensity of the radiation field as traced by the [NeIII]/[NeII] maps. In agreement with recent JWST results, we find a predominance of neutral PAH molecules in the nuclei of Active Galaxies and their outflows. Ionised PAHs are the dominant population in star-forming galaxies. We discuss the implications of our findings for the use of PAHs as ISM tracers in high redshift galaxies., Comment: 16 pages, 13 figures, accepted for publication by MNRAS

Polycyclic Aromatic Hydrocarbons (PAHs) are organic molecules which comprise the smallest particles of dust in the interstellar medium (ISM). Due to their broad/complex emission profiles, obtaining kinematics is a challenge with traditional methods, especially before the advent of the JWST. In this work we employ Principal Component Analysis (PCA) tomography to analyse JWST/NIRSpec IFU data of 3 nearby luminous infrared galaxies (LIRGs), namely, NGC 3256 N, NGC 3256 S and NGC 7469. We detect the signature of rotation in the second principal component of the 3.3 $\mu$m PAH feature in all three targets. We construct velocity maps from the principal components for the 3.3 $\mu$m PAH feature, Br$\beta$ (2.625$\mu$m) and molecular hydrogen, H$_2$ 1-0 S(1) (2.12$\mu$m). We find that in each target, the PAHs qualitatively follow the rotation of the galaxy, consistent with the rotational signature derived from Br$\beta$ and H$_2$. There are however some differences, with the PAH rotation in NGC 3256 N appearing at a different position angle, which suggest differences in the motion of the dust as compared to the gas. This kind of analysis opens a new window into this key component of the ISM., Comment: Accepted for publication in MNRAS Letters. Online supplementary material is included

The escape fraction of ionising photons from galaxies ($f_\mathrm{esc}$) is a key parameter for understanding how intergalactic hydrogen became reionised, but it remains mostly unconstrained. Measurements have been limited to the average value in galaxy ensembles and handfuls of individual detections. To help understand which mechanisms govern ionising photon escape, here we infer the distribution of $f_\mathrm{esc}$. We develop a hierarchical Bayesian inference technique to estimate the population distribution of $f_\mathrm{esc}$ from the ratio of Lyman Continuum to non-ionising UV flux measured from broadband photometry. We apply it to a sample of 148 z ~ 3.5 star-forming galaxies from the VANDELS spectroscopic survey. We explore four physically motivated distributions: constant, log-normal, exponential and bimodal, recovering $\langle f_\mathrm{esc} \rangle \approx5\%$ for most models. We find the observations are best described by an exponential $f_\mathrm{esc}$ distribution with scale factor $\mu=0.05^{+0.01}_{-0.02}$. This indicates most galaxies in our sample exhibit very low escape fractions while predicting substantial ionising photon leakage for only a few galaxies, implying a range of optical depths in the ISM and/or time variability in ionising photon escape. We rule out a bimodal distribution at high significance, indicating that a purely bimodal model of ionising photon escape (due to very strong sightline and/or time variability) is not favoured. We compare our recovered exponential distribution with the SPHINX simulations and find that, while the simulation also predicts an exponential-like distribution, it significantly underpredicts our inferred mean. The distribution of $f_\mathrm{esc}$ can be a vital test for simulations in understanding ionising photon leakage and is important to consider to gain a complete picture of reionisation., Comment: 16 pages, 10 figures, main result in Figure 7, submitted to A&A

We compare mock ultraviolet C II and Si II absorption and emission line features generated using a ~10$^9$ $M_\odot$ virtual galaxy with observations of 131 $z~3$ galaxies from the VANDELS survey. We find that the mock spectra reproduce reasonably well a large majority (83%) of the \vandels\ spectra ($\chi^2<2$), but do not resemble the most massive objects ($>10^{10}M_\odot$) which exhibit broad absorption features. Interestingly, the best-matching mock spectra originate from periods of intense star formation in the virtual galaxy, where its luminosity is four times higher than in periods of relative quiescence. Furthermore, for each galaxy, we predict the Lyman Continuum (LyC) escape fractions using the environment of the virtual galaxy. We derive an average escape fraction of 0.01$\pm$0.02, consistent with other estimates from the literature. The predicted escape fractions are tightly correlated with the Lyman-$\alpha$ escape fractions and highly consistent with observed empirical trends. Additionally, galaxies with larger predicted escape fractions exhibit bluer $\beta$ slopes, more Lyman-$\alpha$ flux, and weaker low-ionization absorption lines. Building upon the good agreement between our predictions and observationally established LyC diagnostics, we examine the LyC leakage mechanisms in the simulation. We find that LyC photon leakage is enhanced in directions where the observed flux dominantly emerges from compact regions depleted of neutral gas and dust, mirroring the scenario inferred from observational data. In general, this study further highlights the potential of high-resolution radiation hydrodynamics simulations in analyzing UV absorption and emission line features and providing valuable insights into the LyC leakage of star-forming galaxies., Comment: 15 pages, 10 figures, 0 birds. Accepted for publication in ApJ

Cross-encoder (CE) models which compute similarity by jointly encoding a query-item pair perform better than embedding-based models (dual-encoders) at estimating query-item relevance. Existing approaches perform k-NN search with CE by approximating the CE similarity with a vector embedding space fit either with dual-encoders (DE) or CUR matrix factorization. DE-based retrieve-and-rerank approaches suffer from poor recall on new domains and the retrieval with DE is decoupled from the CE. While CUR-based approaches can be more accurate than the DE-based approach, they require a prohibitively large number of CE calls to compute item embeddings, thus making it impractical for deployment at scale. In this paper, we address these shortcomings with our proposed sparse-matrix factorization based method that efficiently computes latent query and item embeddings to approximate CE scores and performs k-NN search with the approximate CE similarity. We compute item embeddings offline by factorizing a sparse matrix containing query-item CE scores for a set of train queries. Our method produces a high-quality approximation while requiring only a fraction of CE calls as compared to CUR-based methods, and allows for leveraging DE to initialize the embedding space while avoiding compute- and resource-intensive finetuning of DE via distillation. At test time, the item embeddings remain fixed and retrieval occurs over rounds, alternating between a) estimating the test query embedding by minimizing error in approximating CE scores of items retrieved thus far, and b) using the updated test query embedding for retrieving more items. Our k-NN search method improves recall by up to 5% (k=1) and 54% (k=100) over DE-based approaches. Additionally, our indexing approach achieves a speedup of up to 100x over CUR-based and 5x over DE distillation methods, while matching or improving k-NN search recall over baselines., Comment: ICLR 2024