Search

This is an index of the contributions by the Giant Radio Array for Neutrino Detection (GRAND) Collaboration to the 10th International Workshop on Acoustic and Radio EeV Neutrino Detection Activities (ARENA 2024, University of Chicago, June 11-14, 2024). The contributions include an overview of GRAND in its present and future incarnations, methods of radio-detection that are being developed for them, and ongoing joint work between the GRAND and BEACON experiments., Comment: Note: To access the list of contributions, please follow the "HTML" link that can be found on the arXiv page

Fundamental Measure Theory (FMT) is a successful and versatile approach for describing the properties of the hard-sphere fluid and hard-sphere mixtures within the framework of classical density functional theory (DFT). Lutsko [Phys. Rev. E 102, 062137 (2020)] introduced a version of FMT containing two free parameters, to be fixed by additional physical constraints. Whereas Lutsko focused on the stability of crystalline phases, we introduce and employ two statistical mechanical sum rules pertinent for the fluid phase, that are not automatically satisfied by FMT. By minimizing the relative deviation between different routes to calculate the excess chemical potential and the isothermal compressibility we determine the two free parameters of the theory. Our results indicate that requiring consistency with these sum rules can improve the quality of predictions of FMT for properties of the hard-sphere fluid phase. We suggest that employing these (test particle) sum rules, which apply for any interparticle pair-potential, might provide a means of testing the performance and accuracy of general DFT approximations.

One of the major challenges for the radio detection of extensive air showers, as encountered by the Giant Radio Array for Neutrino Detection (GRAND), is the requirement of an autonomous radio self-trigger. This work presents the current development of self-triggering techniques at the detection-unit level -- the so-called first-level trigger (FLT) -- in the context of the NUTRIG project. A second-level trigger (SLT) at the array level is described in a separate contribution. Two FLT methods are described, based on a template-fitting algorithm and a convolutional neural network (CNN). In this work, we compare the preliminary offline performance of both FLT methods in terms of signal selection efficiency and background rejection efficiency. We find that for both methods, ${\gtrsim}40\%$ of the background can be rejected if a signal selection efficiency of 90\% is required at the $5\sigma$ level., Comment: Presented at the 10th International Workshop on Acoustic and Radio EeV Neutrino Activities (ARENA2024). 8 pages, 4 figures

Large language models struggle to synthesize disparate pieces of information into a coherent plan when approaching a complex procedural task. In this work, we introduce a novel formalism and structure for such procedural knowledge. Based on this formalism, we present a novel procedural knowledge dataset called LCStep, which we created from LangChain tutorials. To leverage this procedural knowledge to solve new tasks, we propose analogy-augmented generation (AAG), which draws inspiration from the human ability to assimilate past experiences to solve unfamiliar problems. AAG uses a custom procedure memory store to retrieve and adapt specialized domain knowledge to answer new procedural tasks. We demonstrate that AAG outperforms few-shot and RAG baselines on LCStep, RecipeNLG, and CHAMP datasets under a pairwise LLM-based evaluation, corroborated by human evaluation in the case of RecipeNLG.

PLATO (PLAnetary Transits and Oscillations of stars) is ESA's M3 mission designed to detect and characterise extrasolar planets and perform asteroseismic monitoring of a large number of stars. PLATO will detect small planets (down to <2 R_(Earth)) around bright stars (<11 mag), including terrestrial planets in the habitable zone of solar-like stars. With the complement of radial velocity observations from the ground, planets will be characterised for their radius, mass, and age with high accuracy (5 %, 10 %, 10 % for an Earth-Sun combination respectively). PLATO will provide us with a large-scale catalogue of well-characterised small planets up to intermediate orbital periods, relevant for a meaningful comparison to planet formation theories and to better understand planet evolution. It will make possible comparative exoplanetology to place our Solar System planets in a broader context. In parallel, PLATO will study (host) stars using asteroseismology, allowing us to determine the stellar properties with high accuracy, substantially enhancing our knowledge of stellar structure and evolution. The payload instrument consists of 26 cameras with 12cm aperture each. For at least four years, the mission will perform high-precision photometric measurements. Here we review the science objectives, present PLATO's target samples and fields, provide an overview of expected core science performance as well as a description of the instrument and the mission profile at the beginning of the serial production of the flight cameras. PLATO is scheduled for a launch date end 2026. This overview therefore provides a summary of the mission to the community in preparation of the upcoming operational phases.

Vision-language models (VLMs) excel at tasks requiring joint understanding of visual and linguistic information. A particularly promising yet under-explored application for these models lies in answering questions based on various kinds of maps. This study investigates the efficacy of VLMs in answering questions based on choropleth maps, which are widely used for data analysis and representation. To facilitate and encourage research in this area, we introduce a novel map-based question-answering benchmark, consisting of maps from three geographical regions (United States, India, China), each containing 1000 questions. Our benchmark incorporates 43 diverse question templates, requiring nuanced understanding of relative spatial relationships, intricate map features, and complex reasoning. It also includes maps with discrete and continuous values, encompassing variations in color-mapping, category ordering, and stylistic patterns, enabling comprehensive analysis. We evaluate the performance of multiple VLMs on this benchmark, highlighting gaps in their abilities and providing insights for improving such models., Comment: 30 Pages, 46 Tables, 6 Figure

The Giant Radio Array for Neutrino Detection (GRAND) is a distributed, sparse ground antenna array designed to detect the radio emission from highly inclined extensive air showers induced by ultra-high-energy particles in the atmosphere. We use CoREAS air-shower simulations to adapt an existing signal model of the radio emission of inclined showers to the 50-200 MHz frequency band GRAND is sensitive to. The model uses a parameterisation of the charge excess emission to isolate the geomagnetic component. By fitting a one-dimensional lateral distribution function to the geomagnetic energy fluence of a shower, we reconstruct its radiation energy. This work details the signal model and the intrinsic event reconstruction of our method, as well as the adaptations to the new frequency band. This work is part of the NUTRIG project., Comment: 8 pages, 6 figures, Conference proceeding for 10th International Workshop on Acoustic and Radio EeV Neutrino Detection Activities

Large-scale cosmic-ray detectors like the Giant Radio Array for Neutrino Detection (GRAND) are pushing the boundaries of our ability to identify air shower events. Existing trigger schemes rely solely on the timing of signals detected by individual antennas, which brings many challenges in distinguishing true air shower signals from background. This work explores novel event-level radio trigger methods specifically designed for GRAND, but also applicable to other systems, such as the Radio Detector (RD) of the Pierre Auger Observatory. In addition to an upgraded plane wave front reconstruction technique, we introduce orthogonal and complementary approaches that analyze the radio-emission footprint, the spatial distribution of signal strength across triggered antennas, to refine event selection. We test our methods on mock data sets constructed with simulated showers and real background noise measured with the GRAND prototype, to assess the performance potential in terms of sensitivity and background rejection in GRAND. Our preliminary results are a first step to identifying the most discriminating radio signal features at event-level, and optimizing the techniques for future implementation on experimental data., Comment: 8 pages, 5 figures, 10th International Workshop on Acoustic and Radio EeV Neutrino Detection Activities - ARENA2024

Simple data augmentation techniques, such as rotations and flips, are widely used to enhance the generalization power of computer vision models. However, these techniques often fail to modify high-level semantic attributes of a class. To address this limitation, researchers have explored generative augmentation methods like the recently proposed DA-Fusion. Despite some progress, the variations are still largely limited to textural changes, thus falling short on aspects like varied viewpoints, environment, weather conditions, or even class-level semantic attributes (eg, variations in a dog's breed). To overcome this challenge, we propose DIAGen, building upon DA-Fusion. First, we apply Gaussian noise to the embeddings of an object learned with Textual Inversion to diversify generations using a pre-trained diffusion model's knowledge. Second, we exploit the general knowledge of a text-to-text generative model to guide the image generation of the diffusion model with varied class-specific prompts. Finally, we introduce a weighting mechanism to mitigate the impact of poorly generated samples. Experimental results across various datasets show that DIAGen not only enhances semantic diversity but also improves the performance of subsequent classifiers. The advantages of DIAGen over standard augmentations and the DA-Fusion baseline are particularly pronounced with out-of-distribution samples., Comment: Accepted for publication in GCPR 2024

Multimodal foundation models serve numerous applications at the intersection of vision and language. Still, despite being pretrained on extensive data, they become outdated over time. To keep models updated, research into continual pretraining mainly explores scenarios with either (1) infrequent, indiscriminate updates on large-scale new data, or (2) frequent, sample-level updates. However, practical model deployment often operates in the gap between these two limit cases, as real-world applications often demand adaptation to specific subdomains, tasks or concepts -- spread over the entire, varying life cycle of a model. In this work, we complement current perspectives on continual pretraining through a research test bed as well as provide comprehensive guidance for effective continual model updates in such scenarios. We first introduce FoMo-in-Flux, a continual multimodal pretraining benchmark with realistic compute constraints and practical deployment requirements, constructed over 63 datasets with diverse visual and semantic coverage. Using FoMo-in-Flux, we explore the complex landscape of practical continual pretraining through multiple perspectives: (1) A data-centric investigation of data mixtures and stream orderings that emulate real-world deployment situations, (2) a method-centric investigation ranging from simple fine-tuning and traditional continual learning strategies to parameter-efficient updates and model merging, (3) meta learning rate schedules and mechanistic design choices, and (4) the influence of model and compute scaling. Together, our insights provide a practitioner's guide to continual multimodal pretraining for real-world deployment. Our benchmark and code is here: https://github.com/ExplainableML/fomo_in_flux., Comment: Technical Report. 52 pages

Controlling scattering and routing of chiral light at the nanoscale is important for optical information processing and imaging, quantum technologies as well as optical manipulation. Here, we introduce a concept of rotating chiral dipoles in order to achieve unidirectional chiral scattering. Implementing this concept by engineering multipole excitations in plasmonic helicoidal nanoparticles, we experimentally demonstrate enantio-sensitive and highly-directional forward scattering of circularly polarised light. The intensity of this highly-directional scattering is defined by the mutual relation between the handedness of the incident light and the chirality of the structure. The concept of rotating chiral dipoles opens up numerous possibilities for engineering of scattering from chiral nanostructures and optical nano-antennas for the design and application of chiral light-matter interaction., Comment: 17 pages,4 figures

The increasing difficulty to distinguish language-model-generated from human-written text has led to the development of detectors of machine-generated text (MGT). However, in many contexts, a black-box prediction is not sufficient, it is equally important to know on what grounds a detector made that prediction. Explanation methods that estimate feature importance promise to provide indications of which parts of an input are used by classifiers for prediction. However, the quality of different explanation methods has not previously been assessed for detectors of MGT. This study conducts the first systematic evaluation of explanation quality for this task. The dimensions of faithfulness and stability are assessed with five automated experiments, and usefulness is evaluated in a user study. We use a dataset of ChatGPT-generated and human-written documents, and pair predictions of three existing language-model-based detectors with the corresponding SHAP, LIME, and Anchor explanations. We find that SHAP performs best in terms of faithfulness, stability, and in helping users to predict the detector's behavior. In contrast, LIME, perceived as most useful by users, scores the worst in terms of user performance at predicting the detectors' behavior.

Existing datasets for tabular question answering typically focus exclusively on text within cells. However, real-world data is inherently multimodal, often blending images such as symbols, faces, icons, patterns, and charts with textual content in tables. With the evolution of AI models capable of multimodal reasoning, it is pertinent to assess their efficacy in handling such structured data. This study investigates whether current AI models can perform knowledge-aware reasoning on multimodal structured data. We explore their ability to reason on tables that integrate both images and text, introducing MMTabQA, a new dataset designed for this purpose. Our experiments highlight substantial challenges for current AI models in effectively integrating and interpreting multiple text and image inputs, understanding visual context, and comparing visual content across images. These findings establish our dataset as a robust benchmark for advancing AI's comprehension and capabilities in analyzing multimodal structured data.

We conducted an experiment during the review process of the 2023 International Conference on Machine Learning (ICML) that requested authors with multiple submissions to rank their own papers based on perceived quality. We received 1,342 rankings, each from a distinct author, pertaining to 2,592 submissions. In this paper, we present an empirical analysis of how author-provided rankings could be leveraged to improve peer review processes at machine learning conferences. We focus on the Isotonic Mechanism, which calibrates raw review scores using author-provided rankings. Our analysis demonstrates that the ranking-calibrated scores outperform raw scores in estimating the ground truth ``expected review scores'' in both squared and absolute error metrics. Moreover, we propose several cautious, low-risk approaches to using the Isotonic Mechanism and author-provided rankings in peer review processes, including assisting senior area chairs' oversight of area chairs' recommendations, supporting the selection of paper awards, and guiding the recruitment of emergency reviewers. We conclude the paper by addressing the study's limitations and proposing future research directions., Comment: See more details about the experiment at https://openrank.cc/

Jahn-Teller distortions of transition-metal coordination environments link orbital occupancies to structure. In the solid state, such distortions can be strongly correlated through the propagation of strain and/or through orbital interactions. Cooperative Jahn-Teller (CJT) order of this kind affects the electronic, magnetic, and structural properties of the materials in which it occurs. Conventionally studied in dense ceramics, CJT order also occurs in hybrid materials, albeit the underlying phenomenology is not well established. Here we use synchrotron powder X-ray diffraction measurements to determine links between composition and Jahn-Teller effects in a series of Prussian blue analogue (PBA) families. We develop a simple model based on the dual considerations of strain and crystal-field stabilisation that can account for the stability, extent of CJT order, and crystallite strain measured experimentally. This model shows how PBA compositions might be tuned to control the emergence and nature of CJT effects. A key implication of the model is that PBAs harbour their own complex interplay between compositional, structural, orbital, and magnetic degrees of freedom related to that of strongly-correlated functional materials such as the manganite perovskites.

The take-home message of this paper is that solving optimal control problems can be computationally straightforward, provided that differentiable partial differential equation (PDE) solvers are available. Although this might seem to be a strong limitation and the development of differentiable PDE solvers might seem arduous, for many problems this is not the case. In particular, for linear partial differential equations, they are equivalent to a linear equation system. Therefore it is just sufficient to be able to solve linear equation systems in an automatic differentiation-capable library like PyTorch and be able to differentiate through the linear solver. Using open-source libraries, like torch_sparse_solve, we have easy to use differentiable direct solvers (here: KLU) at hand, which makes solving linear PDEs straightforward. For nonlinear PDEs, the approach above might be not sufficient, since we do not have an equivalent linear equation system for the underlying PDE. In this case, we can then use the library torch-fenics, which enables us to define PDEs using the finite element library FEniCS and then differentiate through this problem in PyTorch. We complement the proposed methodology with various optimal control problems constrained by stationary and instationary PDEs, linear and nonlinear PDEs (from Poisson to fluid-structure interaction), space-time and time-stepping formulations, parameter estimation, right-hand side control, initial condition control and boundary condition control, finite difference and finite element discretizations and neural network surrogates. All developments of this work are accompanied with the respective source codes published on GitHub.

Analog quantum simulation allows for assessing static and dynamical properties of strongly correlated quantum systems to high precision. To perform simulations outside the reach of classical computers, accurate and reliable implementations of the anticipated Hamiltonians are required. To achieve those, characterization and benchmarking tools are a necessity. For digital quantum devices, randomized benchmarking can provide a benchmark on the average quality of the implementation of a gate set. In this work, we introduce a versatile framework for randomized analog benchmarking of bosonic and fermionic quantum devices implementing particle number preserving dynamics. The scheme makes use of the restricted operations which are native to analog simulators and other continuous variable systems. Importantly, like randomized benchmarking, it is robust against state preparation and measurement errors. We discuss the scheme's efficiency, derive theoretical performance guarantees and showcase the protocol with numerical examples., Comment: Main text 5 pages and 3 figures, references and supplementary material 16 pages and 5 figures

The operation of upcoming ultra-high-energy cosmic-ray, gamma-ray, and neutrino radio-detection experiments, like the Giant Radio Array for Neutrino Detection (GRAND), poses significant computational challenges involving the production of numerous simulations of particle showers and their detection, and a high data throughput. GRANDlib is an open-source software tool designed to meet these challenges. Its primary goal is to perform end-to-end simulations of the detector operation, from the interaction of ultra-high-energy particles, through -- by interfacing with external air-shower simulations -- the ensuing particle shower development and its radio emission, to its detection by antenna arrays and its processing by data-acquisition systems. Additionally, GRANDlib manages the visualization, storage, and retrieval of experimental and simulated data. We present an overview of GRANDlib to serve as the basis of future GRAND analyses., Comment: 11 pages, 9 figures, plus appendices

We present the results of a molecular survey of long period comets C/2021 A1 (Leonard) and C/2022 E3 (ZTF). Comet C/2021 A1 was observed with the IRAM 30-m radio telescope in November-December 2021 before perihelion when it was closest to the Earth. We observed C/2022 E3 in January-February 2023 with the Odin 1-m space telescope and IRAM 30-m, shortly after its perihelion, and when it was closest to the Earth. Snapshots were obtained during 12-16 November 2021 period for comet C/2021 A1. Spectral surveys were undertaken over the 8-13 December 2021 period for comet C/2021 A1 (8, 16, and 61 GHz bandwidth in the 3 mm, 2 mm, and 1 mm window) and over the 3-7 February 2023 period for comet C/2022 E3 (25 and 61 GHz at 2 and 1mm). We report detections of 14 molecular species (HCN, HNC, CH3CN, HNCO, NH2CHO, CH3OH, H2CO, HCOOH, CH3CHO, H2S, CS, OCS, C2H5OH and aGg-(CH2OH)2 ) in both comets plus HC3N and CH2OHCHO marginally detected in C/2021 A1 and CO and H2O (with Odin detected in C/2022 E3. The spatial distribution of several species is investigated. Significant upper limits on the abundances of other molecules and isotopic ratios are also presented. The activity of comet C/2021 A1 did not vary significantly between 13 November and 13 December 2021. Short-term variability in the outgassing of comet C/2022 E3 on the order of +-20% is present and possibly linked to its 8h rotation period. Both comets exhibit rather low abundances relative to water for volatiles species such as CO (< 2%) and H2S (0.15%). Methanol is also rather depleted in comet C/2021 A1 (0.9%). Following their revised photo-destruction rates, HNCO and HCOOH abundances in comets have been reevaluated. Both molecules are relatively enriched in these two comets (0.2% relative to water). We cannot exclude that these species could be produced by the dissociation of ammonium salts., Comment: To be published in Astronomy & Astrophysics

Nested sampling (NS) is a stochastic method for computing the log-evidence of a Bayesian problem. It relies on stochastic estimates of prior volumes enclosed by likelihood contours, which limits the accuracy of the log-evidence calculation. We propose to transform the prior volume estimation into a Bayesian inference problem, which allows us to incorporate a smoothness assumption for likelihood-prior volume relations. As a result, we aim to increase the accuracy of the volume estimates and thus improve the overall log-evidence calculation using NS. The method presented works as a post-processing step for NS and provides posterior samples of the likelihood-prior-volume relation, from which the log-evidence can be calculated. We demonstrate an implementation of the algorithm and compare its results with plain NS on two synthetic datasets for which the underlying evidence is known. We find a significant improvement in accuracy for runs with less than one hundred active samples in NS, but are prone to numerical problems beyond this point.

Phishing websites are a common phenomenon among Tor onion services, and phishers exploit that it is tremendously difficult to distinguish phishing from authentic onion domain names. Operators of onion services devised several strategies to protect their users against phishing. But as we show in this work, none protect users against phishing without producing traces about visited services - something that particularly vulnerable users might want to avoid. In search of a solution we review prior research addressing this problem, and find that only two known approaches, hash visualization and PAKE, are capable of solving this problem. Hash visualization requires users to recognize large hash values. In order to make hash visualization more practical we design a novel mechanism called recognizer, which substantially reduces the amount of information that users must recognize. We analyze the security and privacy properties of our system formally, and report on our prototype implementation as a browser extension for the Tor web browser., Comment: 17 pages, 3 figures

This works investigates the generalization capabilities of MeshGraphNets (MGN) [Pfaff et al. Learning Mesh-Based Simulation with Graph Networks. ICML 2021] to unseen geometries for fluid dynamics, e.g. predicting the flow around a new obstacle that was not part of the training data. For this purpose, we create a new benchmark dataset for data-driven computational fluid dynamics (CFD) which extends DeepMind's flow around a cylinder dataset by including different shapes and multiple objects. We then use this new dataset to extend the generalization experiments conducted by DeepMind on MGNs by testing how well an MGN can generalize to different shapes. In our numerical tests, we show that MGNs can sometimes generalize well to various shapes by training on a dataset of one obstacle shape and testing on a dataset of another obstacle shape.

Results are presented for the measurement of large-scale anisotropies in the arrival directions of ultra-high-energy cosmic rays detected at the Pierre Auger Observatory during 19 years of operation, prior to AugerPrime, the upgrade of the Observatory. The 3D dipole amplitude and direction are reconstructed above $4\,$EeV in four energy bins. Besides the established dipolar anisotropy in right ascension above $8\,$EeV, the Fourier amplitude of the $8$ to $16\,$EeV energy bin is now also above the $5\sigma$ discovery level. No time variation of the dipole moment above $8\,$EeV is found, setting an upper limit to the rate of change of such variations of $0.3\%$ per year at the $95\%$ confidence level. Additionally, the results for the angular power spectrum are shown, demonstrating no other statistically significant multipoles. The results for the equatorial dipole component down to $0.03\,$EeV are presented, using for the first time a data set obtained with a trigger that has been optimized for lower energies. Finally, model predictions are discussed and compared with observations, based on two source emission scenarios obtained in the combined fit of spectrum and composition above $0.6\,$EeV., Comment: Minor modifications, accepted for publication in Astrophysical Journal

Effective routing in satellite mega-constellations has become crucial to facilitate the handling of increasing traffic loads, more complex network architectures, as well as the integration into 6G networks. To enhance adaptability as well as robustness to unpredictable traffic demands, and to solve dynamic routing environments efficiently, machine learning-based solutions are being considered. For network control problems, such as optimizing packet forwarding decisions according to Quality of Service requirements and maintaining network stability, deep reinforcement learning techniques have demonstrated promising results. For this reason, we investigate the viability of multi-agent deep Q-networks for routing in satellite constellation networks. We focus specifically on reward shaping and quantifying training convergence for joint optimization of latency and load balancing in static and dynamic scenarios. To address identified drawbacks, we propose a novel hybrid solution based on centralized learning and decentralized control., Comment: 5 pages, 5 figures, to be published in proceedings of European Space Agency SPAICE Conference 2024, https://spaice.esa.int/

How we perceive objects around us depends on what we actively attend to, yet our eye movements depend on the perceived objects. Still, object segmentation and gaze behavior are typically treated as two independent processes. Drawing on an information processing pattern from robotics, we present a mechanistic model that simulates these processes for dynamic real-world scenes. Our image-computable model uses the current scene segmentation for object-based saccadic decision-making while using the foveated object to refine its scene segmentation recursively. To model this refinement, we use a Bayesian filter, which also provides an uncertainty estimate for the segmentation that we use to guide active scene exploration. We demonstrate that this model closely resembles observers' free viewing behavior, measured by scanpath statistics, including foveation duration and saccade amplitude distributions used for parameter fitting and higher-level statistics not used for fitting. These include how object detections, inspections, and returns are balanced and a delay of returning saccades without an explicit implementation of such temporal inhibition of return. Extensive simulations and ablation studies show that uncertainty promotes balanced exploration and that semantic object cues are crucial to form the perceptual units used in object-based attention. Moreover, we show how our model's modular design allows for extensions, such as incorporating saccadic momentum or pre-saccadic attention, to further align its output with human scanpaths., Comment: 35+16 pages, 8+4 figures

Controlled long-range transport of micro- and nano-scale objects is a key requirement in lab-on-a-chip and microfluidic applications, enabling the efficient capture, concentration, manipulation, and detection of analytes. Traditional methods such as microfluidic pumps and optical trapping face challenges including high power consumption and limited range of action. This study introduces a plasmonic Brownian ratchet designed for the directed transport of dielectric nanometer-sized particles at low optical powers. Through numerical simulations, the ratchet geometry was optimized to enhance electric fields, optical forces, and trapping potentials. Experimentally, the plasmonic ratchet demonstrated the ability to rectify random thermal motion of 40 nm polysterene spheres over extended distances in a specific direction, achieving velocities up to 2.4 $\mu$m/s at excitation powers as low as 0.785 kW/cm^2. This plasmonic ratchet offers a robust and efficient solution for the targeted delivery and concentration of nanoscale analytes on chips, with significant implications for advancing applications in the life sciences., Comment: to be submitted to ACS Nano

Slot attention aims to decompose an input image into a set of meaningful object files (slots). These latent object representations enable various downstream tasks. Yet, these slots often bind to object parts, not objects themselves, especially for real-world datasets. To address this, we introduce Guided Latent Slot Diffusion - GLASS, an object-centric model that uses generated captions as a guiding signal to better align slots with objects. Our key insight is to learn the slot-attention module in the space of generated images. This allows us to repurpose the pre-trained diffusion decoder model, which reconstructs the images from the slots, as a semantic mask generator based on the generated captions. GLASS learns an object-level representation suitable for multiple tasks simultaneously, e.g., segmentation, image generation, and property prediction, outperforming previous methods. For object discovery, GLASS achieves approx. a +35% and +10% relative improvement for mIoU over the previous state-of-the-art (SOTA) method on the VOC and COCO datasets, respectively, and establishes a new SOTA FID score for conditional image generation amongst slot-attention-based methods. For the segmentation task, GLASS surpasses SOTA weakly-supervised and language-based segmentation models, which were specifically designed for the task., Comment: Project Page: https://guided-sa.github.io

One of the major aspects contributing to the striking performance of large language models (LLMs) is the vast amount of factual knowledge accumulated during pre-training. Yet, many LLMs suffer from self-inconsistency, which raises doubts about their trustworthiness and reliability. This paper focuses on entity type ambiguity, analyzing the proficiency and consistency of state-of-the-art LLMs in applying factual knowledge when prompted with ambiguous entities. To do so, we propose an evaluation protocol that disentangles knowing from applying knowledge, and test state-of-the-art LLMs on 49 ambiguous entities. Our experiments reveal that LLMs struggle with choosing the correct entity reading, achieving an average accuracy of only 85%, and as low as 75% with underspecified prompts. The results also reveal systematic discrepancies in LLM behavior, showing that while the models may possess knowledge, they struggle to apply it consistently, exhibit biases toward preferred readings, and display self-inconsistencies. This highlights the need to address entity ambiguity in the future for more trustworthy LLMs., Comment: EMNLP 2024 Findings

Temporal reasoning over tabular data presents substantial challenges for large language models (LLMs), as evidenced by recent research. In this study, we conduct a comprehensive analysis of temporal datasets to pinpoint the specific limitations of LLMs. Our investigation leads to enhancements in TempTabQA, a dataset specifically designed for tabular temporal question answering. We provide critical insights for improving LLM performance in temporal reasoning tasks with tabular data. Furthermore, we introduce a novel approach, C.L.E.A.R to strengthen LLM capabilities in this domain. Our findings demonstrate that our method significantly improves evidence-based reasoning across various models. Additionally, our experimental results reveal that indirect supervision with auxiliary data substantially boosts model performance in these tasks. This work contributes to a deeper understanding of LLMs' temporal reasoning abilities over tabular data and promotes advancements in their application across diverse fields., Comment: Total Pages 18, Total Tables 6, Total figures 7

Self-calibration methods with the CLEAN algorithm have been widely employed in Very Long Baseline Interferometry (VLBI) data processing in order to correct antenna-based amplitude and phase corruptions present in the data. However, human interaction during the conventional CLEAN self-calibration process can impose a strong effective prior, which in turn may produce artifacts within the final image and hinder the reproducibility of final results. In this work, we aim to demonstrate a combined self-calibration and imaging method for VLBI data in a Bayesian inference framework. The method corrects for amplitude and phase gains for each antenna and polarization mode by inferring the temporal correlation of the gain solutions. We use Stokes I data of M87 taken with the Very Long Baseline Array (VLBA) at 43GHz, pre-calibrated using the rPICARD CASA-based pipeline. For antenna-based gain calibration and imaging, we use the Bayesian imaging software resolve. To estimate gain and image uncertainties, we use a Variational Inference method. We obtain a high-resolution M87 Stokes I image at 43GHz in conjunction with antenna-based gain solutions using our Bayesian self-calibration and imaging method. The core with counter-jet structure is better resolved, and extended jet emission is better described compared to the CLEAN reconstruction. Furthermore, uncertainty estimation of the image and antenna-based gains allows us to quantify the reliability of the result. Our Bayesian self-calibration and imaging method is able to reconstruct robust and reproducible Stokes I images and gain solutions with uncertainty estimation by taking into account the uncertainty information in the data., Comment: accepted for publication in A&A

The first generation of ELT instruments includes an optical-infrared high-resolution spectrograph, indicated as ELT-HIRES and recently christened ANDES (ArmazoNes high Dispersion Echelle Spectrograph). ANDES consists of three fibre-fed spectrographs ([U]BV, RIZ, YJH) providing a spectral resolution of $\sim$100,000 with a minimum simultaneous wavelength coverage of 0.4-1.8 $\mu$m with the goal of extending it to 0.35-2.4 $\mu$m with the addition of a U arm to the BV spectrograph and a separate K band spectrograph. It operates both in seeing- and diffraction-limited conditions and the fibre feeding allows several, interchangeable observing modes including a single conjugated adaptive optics module and a small diffraction-limited integral field unit in the NIR. Modularity and fibre-feeding allow ANDES to be placed partly on the ELT Nasmyth platform and partly in the Coud\'e room. ANDES has a wide range of groundbreaking science cases spanning nearly all areas of research in astrophysics and even fundamental physics. Among the top science cases, there are the detection of biosignatures from exoplanet atmospheres, finding the fingerprints of the first generation of stars, tests on the stability of Nature's fundamental couplings, and the direct detection of the cosmic acceleration. The ANDES project is carried forward by a large international consortium, composed of 35 Institutes from 13 countries, forming a team of almost 300 scientists and engineers which include the majority of the scientific and technical expertise in the field that can be found in ESO member states., Comment: SPIE astronomical telescope and instrumentation 2024, in press

Deploying deep learning-based imaging tools across various clinical sites poses significant challenges due to inherent domain shifts and regulatory hurdles associated with site-specific fine-tuning. For histopathology, stain normalization techniques can mitigate discrepancies, but they often fall short of eliminating inter-site variations. Therefore, we present Data Alchemy, an explainable stain normalization method combined with test time data calibration via a template learning framework to overcome barriers in cross-site analysis. Data Alchemy handles shifts inherent to multi-site data and minimizes them without needing to change the weights of the normalization or classifier networks. Our approach extends to unseen sites in various clinical settings where data domain discrepancies are unknown. Extensive experiments highlight the efficacy of our framework in tumor classification in hematoxylin and eosin-stained patches. Our explainable normalization method boosts classification tasks' area under the precision-recall curve(AUPR) by 0.165, 0.545 to 0.710. Additionally, Data Alchemy further reduces the multisite classification domain gap, by improving the 0.710 AUPR an additional 0.142, elevating classification performance further to 0.852, from 0.545. Our Data Alchemy framework can popularize precision medicine with minimal operational overhead by allowing for the seamless integration of pre-trained deep learning-based clinical tools across multiple sites., Comment: accepted to Machine Learning in Medical Imaging (MLMI 2024)

We tackle the task of text-to-speech (TTS) in Hebrew. Traditional Hebrew contains Diacritics, which dictate the way individuals should pronounce given words, however, modern Hebrew rarely uses them. The lack of diacritics in modern Hebrew results in readers expected to conclude the correct pronunciation and understand which phonemes to use based on the context. This imposes a fundamental challenge on TTS systems to accurately map between text-to-speech. In this work, we propose to adopt a language modeling Diacritics-Free approach, for the task of Hebrew TTS. The model operates on discrete speech representations and is conditioned on a word-piece tokenizer. We optimize the proposed method using in-the-wild weakly supervised data and compare it to several diacritic-based TTS systems. Results suggest the proposed method is superior to the evaluated baselines considering both content preservation and naturalness of the generated speech. Samples can be found under the following link: pages.cs.huji.ac.il/adiyoss-lab/HebTTS/, Comment: Accepted at Interspeech24

Attribution maps are one of the most established tools to explain the functioning of computer vision models. They assign importance scores to input features, indicating how relevant each feature is for the prediction of a deep neural network. While much research has gone into proposing new attribution methods, their proper evaluation remains a difficult challenge. In this work, we propose a novel evaluation protocol that overcomes two fundamental limitations of the widely used incremental-deletion protocol, i.e., the out-of-domain issue and lacking inter-model comparisons. This allows us to evaluate 23 attribution methods and how eight different design choices of popular vision models affect their attribution quality. We find that intrinsically explainable models outperform standard models and that raw attribution values exhibit a higher attribution quality than what is known from previous work. Further, we show consistent changes in the attribution quality when varying the network design, indicating that some standard design choices promote attribution quality.

Graph convolutions have gained popularity due to their ability to efficiently operate on data with an irregular geometric structure. However, graph convolutions cause over-smoothing, which refers to representations becoming more similar with increased depth. However, many different definitions and intuitions currently coexist, leading to research efforts focusing on incompatible directions. This paper attempts to align these directions by showing that over-smoothing is merely a special case of power iteration. This greatly simplifies the existing theory on over-smoothing, making it more accessible. Based on the theory, we provide a novel comprehensive definition of rank collapse as a generalized form of over-smoothing and introduce the rank-one distance as a corresponding metric. Our empirical evaluation of 14 commonly used methods shows that more models than were previously known suffer from this issue.

Chart question answering (CQA) is a crucial area of Visual Language Understanding. However, the robustness and consistency of current Visual Language Models (VLMs) in this field remain under-explored. This paper evaluates state-of-the-art VLMs on comprehensive datasets, developed specifically for this study, encompassing diverse question categories and chart formats. We investigate two key aspects: 1) the models' ability to handle varying levels of chart and question complexity, and 2) their robustness across different visual representations of the same underlying data. Our analysis reveals significant performance variations based on question and chart types, highlighting both strengths and weaknesses of current models. Additionally, we identify areas for improvement and propose future research directions to build more robust and reliable CQA systems. This study sheds light on the limitations of current models and paves the way for future advancements in the field., Comment: 22 pages, 9 Tables, 5 figures, 22 examples

Cognitive textual and visual reasoning tasks, such as puzzles, series, and analogies, demand the ability to quickly reason, decipher, and evaluate patterns both textually and spatially. While LLMs and VLMs, through extensive training on large amounts of human-curated data, have attained a high level of pseudo-human intelligence in some common sense reasoning tasks, they still struggle with more complex reasoning tasks that require cognitive understanding. In this work, we introduce a new dataset, NTSEBench, designed to evaluate the cognitive multi-modal reasoning and problem-solving skills of large models. The dataset comprises 2,728 multiple-choice questions comprising of a total of 4,642 images across 26 categories sampled from the NTSE examination conducted nationwide in India, featuring both visual and textual general aptitude questions that do not rely on rote learning. We establish baselines on the dataset using state-of-the-art LLMs and VLMs. To facilitate a comparison between open source and propriety models, we propose four distinct modeling strategies to handle different modalities (text and images) in the dataset instances., Comment: 15 pages, 2 figures, 5 tables

*Multi-Instance Partial Label Learning* (MI-PLL) is a weakly-supervised learning setting encompassing *partial label learning*, *latent structural learning*, and *neurosymbolic learning*. Unlike supervised learning, in MI-PLL, the inputs to the classifiers at training-time are tuples of instances $\mathbf{x}$. At the same time, the supervision signal is generated by a function $\sigma$ over the (hidden) gold labels of $\mathbf{x}$. In this work, we make multiple contributions towards addressing a problem that hasn't been studied so far in the context of MI-PLL: that of characterizing and mitigating *learning imbalances*, i.e., major differences in the errors occurring when classifying instances of different classes (aka *class-specific risks*). In terms of theory, we derive class-specific risk bounds for MI-PLL, while making minimal assumptions. Our theory reveals a unique phenomenon: that $\sigma$ can greatly impact learning imbalances. This result is in sharp contrast with previous research on supervised and weakly-supervised learning, which only studies learning imbalances under the prism of data imbalances. On the practical side, we introduce a technique for estimating the marginal of the hidden labels using only MI-PLL data. Then, we introduce algorithms that mitigate imbalances at training- and testing-time, by treating the marginal of the hidden labels as a constraint. We demonstrate the effectiveness of our techniques using strong baselines from neurosymbolic and long-tail learning, suggesting performance improvements of up to 14\%.

Learning disentangled representations from unlabelled data is a fundamental challenge in machine learning. Solving it may unlock other problems, such as generalization, interpretability, or fairness. Although remarkably challenging to solve in theory, disentanglement is often achieved in practice through prior matching. Furthermore, recent works have shown that prior matching approaches can be enhanced by leveraging geometrical considerations, e.g., by learning representations that preserve geometric features of the data, such as distances or angles between points. However, matching the prior while preserving geometric features is challenging, as a mapping that fully preserves these features while aligning the data distribution with the prior does not exist in general. To address these challenges, we introduce a novel approach to disentangled representation learning based on quadratic optimal transport. We formulate the problem using Gromov-Monge maps that transport one distribution onto another with minimal distortion of predefined geometric features, preserving them as much as can be achieved. To compute such maps, we propose the Gromov-Monge-Gap (GMG), a regularizer quantifying whether a map moves a reference distribution with minimal geometry distortion. We demonstrate the effectiveness of our approach for disentanglement across four standard benchmarks, outperforming other methods leveraging geometric considerations.

We present HebDB, a weakly supervised dataset for spoken language processing in the Hebrew language. HebDB offers roughly 2500 hours of natural and spontaneous speech recordings in the Hebrew language, consisting of a large variety of speakers and topics. We provide raw recordings together with a pre-processed, weakly supervised, and filtered version. The goal of HebDB is to further enhance research and development of spoken language processing tools for the Hebrew language. Hence, we additionally provide two baseline systems for Automatic Speech Recognition (ASR): (i) a self-supervised model; and (ii) a fully supervised model. We present the performance of these two methods optimized on HebDB and compare them to current multi-lingual ASR alternatives. Results suggest the proposed method reaches better results than the evaluated baselines considering similar model sizes. Dataset, code, and models are publicly available under https://pages.cs.huji.ac.il/adiyoss-lab/HebDB/., Comment: Accepted at Interspeech2024

Ultra-high-energy cosmic rays are known to be mainly of extragalactic origin, and their propagation is limited by energy losses, so their arrival directions are expected to correlate with the large-scale structure of the local Universe. In this work, we investigate the possible presence of intermediate-scale excesses in the flux of the most energetic cosmic rays from the direction of the supergalactic plane region using events with energies above 20 EeV recorded with the surface detector array of the Pierre Auger Observatory up to 31 December 2022, with a total exposure of 135,000 km^2 sr yr. The strongest indication for an excess that we find, with a post-trial significance of 3.1{\sigma}, is in the Centaurus region, as in our previous reports, and it extends down to lower energies than previously studied. We do not find any strong hints of excesses from any other region of the supergalactic plane at the same angular scale. In particular, our results do not confirm the reports by the Telescope Array collaboration of excesses from two regions in the Northern Hemisphere at the edge of the field of view of the Pierre Auger Observatory. With a comparable exposure, our results in those regions are in good agreement with the expectations from an isotropic distribution., Comment: submitted to ApJ

A diverse range of large language models (LLMs), e.g., ChatGPT, and visual question answering (VQA) models, e.g., BLIP, have been developed for solving textual and visual question answering tasks. However, both LLMs and VQA models encounter challenges when applied to task-specific datasets. Fine-tuning these models is either difficult, as it requires access via APIs, rendering them as black-boxes, or costly due to the need of tuning a large number of parameters. To address this, we introduce InfoSel, a data-efficient and lightweight ensemble method that learns to dynamically pick the winner from existing black-box models for predictions on both textual and multimodal visual question answering tasks. Unlike traditional ensemble models, InfoSel does not rely on prediction probabilities or confidences, which typically are not available in black-box models. Experimental results on four datasets demonstrate that our approach achieves an absolute increase of up to +5.27% in the F1-score compared to standalone LLMs. Remarkably, this improvement is achieved by utilizing only 1K training instances and 110M model parameters for training task-specific ensemble models.

The Planetary Transits and Oscillations of stars mission (PLATO) will allow us to measure surface rotation and monitor photometric activity of tens of thousands of main sequence solar-type and subgiant stars. This paper is the first of a series dedicated to the preparation of the analysis of stellar surface rotation and photospheric activity with the near-future PLATO data. We describe in this work the strategy that will be implemented in the PLATO pipeline to measure stellar surface rotation, photometric activity, and long-term modulations. The algorithms are applied on both noise-free and noisy simulations of solar-type stars, which include activity cycles, latitudinal differential rotation, and spot evolution. PLATO simulated systematics are included in the noisy light curves. We show that surface rotation periods can be recovered with confidence for most of the stars with only six months of observations and that the {recovery rate} of the analysis significantly improves as additional observations are collected. This means that the first PLATO data release will already provide a substantial set of measurements for this quantity, with a significant refinement on their quality as the instrument obtains longer light curves. Measuring the Schwabe-like magnetic activity cycle during the mission will require that the same field be observed over a significant timescale (more than four years). Nevertheless, PLATO will provide a vast and robust sample of solar-type stars with constraints on the activity-cycle length. Such a sample is lacking from previous missions dedicated to space photometry., Comment: 19 pages, 16 figures, accepted for publication in A&A

In digital pathology, the traditional method for deep learning-based image segmentation typically involves a two-stage process: initially segmenting high-resolution whole slide images (WSI) into smaller patches (e.g., 256x256, 512x512, 1024x1024) and subsequently reconstructing them to their original scale. This method often struggles to capture the complex details and vast scope of WSIs. In this paper, we propose the holistic histopathology (HoloHisto) segmentation method to achieve end-to-end segmentation on gigapixel WSIs, whose maximum resolution is above 80,000$\times$70,000 pixels. HoloHisto fundamentally shifts the paradigm of WSI segmentation to an end-to-end learning fashion with 1) a large (4K) resolution base patch for elevated visual information inclusion and efficient processing, and 2) a novel sequential tokenization mechanism to properly model the contextual relationships and efficiently model the rich information from the 4K input. To our best knowledge, HoloHisto presents the first holistic approach for gigapixel resolution WSI segmentation, supporting direct I/O of complete WSI and their corresponding gigapixel masks. Under the HoloHisto platform, we unveil a random 4K sampler that transcends ultra-high resolution, delivering 31 and 10 times more pixels than standard 2D and 3D patches, respectively, for advancing computational capabilities. To facilitate efficient 4K resolution dense prediction, we leverage sequential tokenization, utilizing a pre-trained image tokenizer to group image features into a discrete token grid. To assess the performance, our team curated a new kidney pathology image segmentation (KPIs) dataset with WSI-level glomeruli segmentation from whole mouse kidneys. From the results, HoloHisto-4K delivers remarkable performance gains over previous state-of-the-art models.