Search

Visual Question Answering (VQA) has emerged as a promising area of research to develop AI-based systems for enabling interactive and immersive learning. Numerous VQA datasets have been introduced to facilitate various tasks, such as answering questions or identifying unanswerable ones. However, most of these datasets are constructed using real-world images, leaving the performance of existing models on cartoon images largely unexplored. Hence, in this paper, we present "SimpsonsVQA", a novel dataset for VQA derived from The Simpsons TV show, designed to promote inquiry-based learning. Our dataset is specifically designed to address not only the traditional VQA task but also to identify irrelevant questions related to images, as well as the reverse scenario where a user provides an answer to a question that the system must evaluate (e.g., as correct, incorrect, or ambiguous). It aims to cater to various visual applications, harnessing the visual content of "The Simpsons" to create engaging and informative interactive systems. SimpsonsVQA contains approximately 23K images, 166K QA pairs, and 500K judgments (https://simpsonsvqa.org). Our experiments show that current large vision-language models like ChatGPT4o underperform in zero-shot settings across all three tasks, highlighting the dataset's value for improving model performance on cartoon images. We anticipate that SimpsonsVQA will inspire further research, innovation, and advancements in inquiry-based learning VQA.

Metaheuristic search methods have proven to be essential tools for tackling complex optimization challenges, but their full potential is often constrained by conventional algorithmic frameworks. In this paper, we introduce a novel approach called Deep Heuristic Search (DHS), which models metaheuristic search as a memory-driven process. DHS employs multiple search layers and memory-based exploration-exploitation mechanisms to navigate large, dynamic search spaces. By utilizing model-free memory representations, DHS enhances the ability to traverse temporal trajectories without relying on probabilistic transition models. The proposed method demonstrates significant improvements in search efficiency and performance across a range of heuristic optimization problems., Comment: 10 pages, 6 figures

Synthetic data has gained attention for training large language models, but poor-quality data can harm performance (see, e.g., Shumailov et al. (2023); Seddik et al. (2024)). A potential solution is data pruning, which retains only high-quality data based on a score function (human or machine feedback). Previous work Feng et al. (2024) analyzed models trained on synthetic data as sample size increases. We extend this by using random matrix theory to derive the performance of a binary classifier trained on a mix of real and pruned synthetic data in a high dimensional setting. Our findings identify conditions where synthetic data could improve performance, focusing on the quality of the generative model and verification strategy. We also show a smooth phase transition in synthetic label noise, contrasting with prior sharp behavior in infinite sample limits. Experiments with toy models and large language models validate our theoretical results.

In this technical report, we present Falcon Mamba 7B, a new base large language model based on the novel Mamba architecture. Falcon Mamba 7B is trained on 5.8 trillion tokens with carefully selected data mixtures. As a pure Mamba-based model, Falcon Mamba 7B surpasses leading open-weight models based on Transformers, such as Mistral 7B, Llama3.1 8B, and Falcon2 11B. It is on par with Gemma 7B and outperforms models with different architecture designs, such as RecurrentGemma 9B and RWKV-v6 Finch 7B/14B. Currently, Falcon Mamba 7B is the best-performing Mamba model in the literature at this scale, surpassing both existing Mamba and hybrid Mamba-Transformer models, according to the Open LLM Leaderboard. Due to its architecture, Falcon Mamba 7B is significantly faster at inference and requires substantially less memory for long sequence generation. Despite recent studies suggesting that hybrid Mamba-Transformer models outperform pure architecture designs, we demonstrate that even the pure Mamba design can achieve similar, or even superior results compared to the Transformer and hybrid designs. We make the weights of our implementation of Falcon Mamba 7B publicly available on https://huggingface.co/tiiuae/falcon-mamba-7b, under a permissive license.

General relativistic Riemann solvers are typically complex, fragile and unwieldy, at least in comparison to their special relativistic counterparts. In this paper, we present a new high-resolution shock-capturing algorithm on curved spacetimes that employs a local coordinate transformation at each inter-cell boundary, transforming all primitive and conservative variables into a locally flat spacetime coordinate basis (i.e., the tetrad basis), generalizing previous approaches developed for relativistic hydrodynamics. This algorithm enables one to employ a purely special relativistic Riemann solver, combined with an appropriate post-hoc flux correction step, irrespective of the geometry of the underlying Lorentzian manifold. We perform a systematic validation of the algorithm using the Gkeyll simulation framework for both general relativistic electromagnetism and general relativistic hydrodynamics, highlighting the algorithm's superior convergence and stability properties in each case when compared against standard analytical solutions for black hole magnetosphere and ultra-relativistic black hole accretion problems. However, as an illustration of the generality and practicality of the algorithm, we also apply it to more astrophysically realistic magnetosphere and fluid accretion problems in the limit of high black hole spin, for which standard general relativistic Riemann solvers are often too unstable to produce useful solutions., Comment: 30 pages, 25 figures. Prepared for submission for ApJ

The escalating sophistication of Android malware poses significant challenges to traditional detection methods, necessitating innovative approaches that can efficiently identify and classify threats with high precision. This paper introduces a novel framework that synergistically integrates an attention-enhanced Multi-Layer Perceptron (MLP) with a Support Vector Machine (SVM) to make Android malware detection and classification more effective. By carefully analyzing a mere 47 features out of over 9,760 available in the comprehensive CCCS-CIC-AndMal-2020 dataset, our MLP-SVM model achieves an impressive accuracy over 99% in identifying malicious applications. The MLP, enhanced with an attention mechanism, focuses on the most discriminative features and further reduces the 47 features to only 14 components using Linear Discriminant Analysis (LDA). Despite this significant reduction in dimensionality, the SVM component, equipped with an RBF kernel, excels in mapping these components to a high-dimensional space, facilitating precise classification of malware into their respective families. Rigorous evaluations, encompassing accuracy, precision, recall, and F1-score metrics, confirm the superiority of our approach compared to existing state-of-the-art techniques. The proposed framework not only significantly reduces the computational complexity by leveraging a compact feature set but also exhibits resilience against the evolving Android malware landscape., Comment: Accepted for NSS-SocialSec 2024, Lecture Notes in Computer Science (LNCS)

Feeding the growing human population sustainably amidst climate change is one of the most important challenges in the 21st century. Current practices often lead to the overuse of agronomic inputs, such as synthetic fertilizers and water, resulting in environmental contamination and diminishing returns on crop productivity. The complexity of agricultural systems, involving plant-environment interactions and human management, presents significant scientific and technical challenges for developing sustainable practices. Addressing these challenges necessitates transdisciplinary research, involving intense collaboration among fields such as plant science, engineering, computer science, and social sciences. Here, we present five case studies from two research centers demonstrating successful transdisciplinary approaches toward more sustainable water and fertilizer use. These case studies span multiple scales. Starting from whole-plant signaling, we explore how reporter plants can transform our understanding of plant communication and enable efficient application of water and fertilizers. We then show how new fertilizer technologies could increase the availability of phosphorus in the soil. To accelerate advancements in breeding new cultivars, we discuss robotic technologies for high-throughput plant screening in different environments at a population scale. At the ecosystem scale, we investigate phosphorus recovery from aquatic systems and methods to minimize phosphorus leaching. Finally, as agricultural outputs affect all people, we show how to integrate stakeholder perspectives and needs into the research. With these case studies, we hope to encourage the scientific community to adopt transdisciplinary research and promote cross-training among biologists, engineers, and social scientists to drive discovery and innovation in advancing sustainable agricultural systems.

We demonstrate that preference optimization methods can effectively enhance LLM safety. Applying various alignment techniques to the Falcon 11B model using safety datasets, we achieve a significant boost in global safety score (from $57.64\%$ to $99.90\%$) as measured by LlamaGuard 3 8B, competing with state-of-the-art models. On toxicity benchmarks, average scores in adversarial settings dropped from over $0.6$ to less than $0.07$. However, this safety improvement comes at the cost of reduced general capabilities, particularly in math, suggesting a trade-off. We identify noise contrastive alignment (Safe-NCA) as an optimal method for balancing safety and performance. Our study ultimately shows that alignment techniques can be sufficient for building safe and robust models.

The precision of link-level theoretical performance analysis for emerging wireless communication paradigms is critical. Recent studies have demonstrated the excellent fitting capabilities of the mixture gamma (MG) distribution in representing small-scale fading in outdoor terahertz (THz)-band scenarios. Our study establishes an in-depth performance analysis for outdoor point-to-point THz links under realistic configurations, incorporating MG small-scale fading combined with the misalignment effect. We derive closed-form expressions for the bit-error probability, outage probability, and ergodic capacity. Furthermore, we conduct an asymptotic analysis of these metrics at high signal-to-noise ratios and derive the necessary convergence conditions. Simulation results, leveraging precise measurement-based channel parameters in various configurations, closely align with the derived analytical equations.

As advancements close the gap between current device capabilities and the requirements for terahertz (THz)-band communications, the demand for terabit-per-second (Tbps) circuits is on the rise. This paper addresses the challenge of achieving Tbps data rates in THz-band communications by focusing on the baseband computation bottleneck. We propose leveraging parallel processing and pseudo-soft information (PSI) across multicarrier THz channels for efficient channel code decoding. We map bits to transmission resources using shorter code-words to enhance parallelizability and reduce complexity. Additionally, we integrate channel state information into PSI to alleviate the processing overhead of soft decoding. Results demonstrate that PSI-aided decoding of 64-bit code-words halves the complexity of 128-bit hard decoding under comparable effective rates, while introducing a 4 dB gain at a $10^{-3}$ block error rate. The proposed scheme approximates soft decoding with significant complexity reduction at a graceful performance cost.

The recent surge in AI-generated songs presents exciting possibilities and challenges. While these inventions democratize music creation, they also necessitate the ability to distinguish between human-composed and synthetic songs to safeguard artistic integrity and protect human musical artistry. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI-generated but the instrumental music is sourced from real songs. However, these approaches are inadequate for detecting contemporary end-to-end artificial songs where all components (vocals, music, lyrics, and style) could be AI-generated. Additionally, existing datasets lack music-lyrics diversity, long-duration songs, and open-access fake songs. To address these gaps, we introduce SONICS, a novel dataset for end-to-end Synthetic Song Detection (SSD), comprising over 97k songs (4,751 hours) with over 49k synthetic songs from popular platforms like Suno and Udio. Furthermore, we highlight the importance of modeling long-range temporal dependencies in songs for effective authenticity detection, an aspect entirely overlooked in existing methods. To utilize long-range patterns, we introduce SpecTTTra, a novel architecture that significantly improves time and memory efficiency over conventional CNN and Transformer-based models. In particular, for long audio samples, our top-performing variant outperforms ViT by 8% F1 score while being 38% faster and using 26% less memory. Additionally, in comparison with ConvNeXt, our model achieves 1% gain in F1 score with 20% boost in speed and 67% reduction in memory usage. Other variants of our model family provide even better speed and memory efficiency with competitive performance., Comment: Updated with correction

During 2023, two interesting results were proven about the limit behavior of game dynamics: First, it was shown that there is a game for which no dynamics converges to the Nash equilibria. Second, it was shown that the sink equilibria of a game adequately capture the limit behavior of natural game dynamics. These two results have created a need and opportunity to articulate a principled computational theory of the meaning of the game that is based on game dynamics. Given any game in normal form, and any prior distribution of play, we study the problem of computing the asymptotic behavior of a class of natural dynamics called the noisy replicator dynamics as a limit distribution over the sink equilibria of the game. When the prior distribution has pure strategy support, we prove this distribution can be computed efficiently, in near-linear time to the size of the best-response graph. When the distribution can be sampled -- for example, if it is the uniform distribution over all mixed strategy profiles -- we show through experiments that the limit distribution of reasonably large games can be estimated quite accurately through sampling and simulation.

Stroke remains a leading cause of global morbidity and mortality, placing a heavy socioeconomic burden. Over the past decade, advances in endovascular reperfusion therapy and the use of CT and MRI imaging for treatment guidance have significantly improved patient outcomes and are now standard in clinical practice. To develop machine learning algorithms that can extract meaningful and reproducible models of brain function for both clinical and research purposes from stroke images - particularly for lesion identification, brain health quantification, and prognosis - large, diverse, and well-annotated public datasets are essential. While only a few datasets with (sub-)acute stroke data were previously available, several large, high-quality datasets have recently been made publicly accessible. However, these existing datasets include only MRI data. In contrast, our dataset is the first to offer comprehensive longitudinal stroke data, including acute CT imaging with angiography and perfusion, follow-up MRI at 2-9 days, as well as acute and longitudinal clinical data up to a three-month outcome. The dataset includes a training dataset of n = 150 and a test dataset of n = 100 scans. Training data is publicly available, while test data will be used exclusively for model validation. We are making this dataset available as part of the 2024 edition of the Ischemic Stroke Lesion Segmentation (ISLES) challenge (https://www.isles-challenge.org/), which continuously aims to establish benchmark methods for acute and sub-acute ischemic stroke lesion segmentation, aiding in creating open stroke imaging datasets and evaluating cutting-edge image processing algorithms.

Accurate estimation of core (irreversibly damaged tissue) and penumbra (salvageable tissue) volumes is essential for ischemic stroke treatment decisions. Perfusion CT, the clinical standard, estimates these volumes but is affected by variations in deconvolution algorithms, implementations, and thresholds. Core tissue expands over time, with growth rates influenced by thrombus location, collateral circulation, and inherent patient-specific factors. Understanding this tissue growth is crucial for determining the need to transfer patients to comprehensive stroke centers, predicting the benefits of additional reperfusion attempts during mechanical thrombectomy, and forecasting final clinical outcomes. This work presents the ISLES'24 challenge, which addresses final post-treatment stroke infarct prediction from pre-interventional acute stroke imaging and clinical data. ISLES'24 establishes a unique 360-degree setting where all feasibly accessible clinical data are available for participants, including full CT acute stroke imaging, sub-acute follow-up MRI, and clinical tabular data. The contributions of this work are two-fold: first, we introduce a standardized benchmarking of final stroke infarct segmentation algorithms through the ISLES'24 challenge; second, we provide insights into infarct segmentation using multimodal imaging and clinical data strategies by identifying outperforming methods on a finely curated dataset. The outputs of this challenge are anticipated to enhance clinical decision-making and improve patient outcome predictions. All ISLES'24 materials, including data, performance evaluation scripts, and leading algorithmic strategies, are available to the research community following \url{https://isles-24.grand-challenge.org/}.

Fairness in clustering has been considered extensively in the past; however, the trade-off between the two objectives -- e.g., can we sacrifice just a little in the quality of the clustering to significantly increase fairness, or vice-versa? -- has rarely been addressed. We introduce novel algorithms for tracing the complete trade-off curve, or Pareto front, between quality and fairness in clustering problems; that is, computing all clusterings that are not dominated in both objectives by other clusterings. Unlike previous work that deals with specific objectives for quality and fairness, we deal with all objectives for fairness and quality in two general classes encompassing most of the special cases addressed in previous work. Our algorithm must take exponential time in the worst case as the Pareto front itself can be exponential. Even when the Pareto front is polynomial, our algorithm may take exponential time, and we prove that this is inevitable unless P = NP. However, we also present a new polynomial-time algorithm for computing the entire Pareto front when the cluster centers are fixed, and for perhaps the most natural fairness objective: minimizing the sum, over all clusters, of the imbalance between the two groups in each cluster.

We present the design and observations of low resolution JWST/NIRSpec PRISM spectroscopy from the Ultradeep NIRSpec and NIRCam ObserVations before the Epoch of Reionization (UNCOVER) Cycle 1 JWST Treasury program. Targets are selected using JWST/NIRCam photometry from UNCOVER and other programs, and cover a wide range of categories and redshifts to ensure the legacy value of the survey. These categories include the first galaxies at $z\gtrsim10$, faint galaxies during the Epoch of Reionization ($z\sim6-8$), high redshift AGN ($z\gtrsim6$), Population III star candidates, distant quiescent and dusty galaxies ($1\lesssim z \lesssim 6$), and filler galaxies sampling redshift--color--magnitude space from $z\sim 0.1-13$. Seven NIRSpec MSA masks across the extended Abell 2744 cluster were observed, along with NIRCam parallel imaging in 8 filters (F090W, F115W, F150W, F200W, F277W, F356W, F410M, F444W, F480M) over a total area of ~26 arcmin$^2$, overlapping existing HST coverage from programs including the Hubble Frontier Fields and BUFFALO. We successfully observed 553 objects down to $m_{\mathrm{F444W}}\sim30\mathrm{AB}$, and by leveraging mask overlaps, we reach total on-target exposure times ranging from 2.4-16.7h. We demonstrate the success rate and distribution of confirmed redshifts, and also highlight the rich information revealed by these ultradeep spectra for a subset of our targets. An updated lens model of Abell 2744 is also presented, including 14 additional spectroscopic redshifts and finding a total cluster mass of $M_{\mathrm{SL}}=(2.1\pm0.3)\times10^{15}\,\mathrm{M}_{\odot}$. We publicly release reduced 1D and 2D spectra for all objects observed in Summer 2023 along with a spectroscopic redshift catalog and the updated lens model of the cluster (https://jwst-uncover.github.io/DR4.html)., Comment: 19 pages, 9 figures, 4 tables, submitted to ApJ, comments welcome! Data available at: https://jwst-uncover.github.io/DR4.html (v2: figure format correction)

This article establishes a proof of dynamical localization for a random scattering zipper model. The scattering zipper operator is the product of two unitary by blocks operators, multiplicatively perturbed on the left and right by random unitary phases. One of the operator is shifted so that this configuration produces a random 5-diagonal unitary operator per blocks. To prove the dynamical localization for this operator, we use the method of fractional moments. We first prove the continuity and strict positivity of the Lyapunov exponents in an annulus around the unit circle, which leads to the exponential decay of a power of the norm of the products of transfer matrices. We then establish an explicit formula of the coefficients of the finite resolvent in terms of the coefficients of the transfer matrices using Schur's complement. From this we deduce, through two reduction results, the exponential decay of the resolvent, from which we get the dynamical localization.

The mass-metallicity relation (MZR) provides crucial insights into the baryon cycle in galaxies and provides strong constraints on galaxy formation models. We use JWST NIRSpec observations from the UNCOVER program to measure the gas-phase metallicity in a sample of eight galaxies during the epoch of reionization at $z=6-8$. Thanks to strong lensing of the galaxy cluster Abell 2744, we are able to probe extremely low stellar masses between $10^{6}$ and $10^{8} M_{\odot}$. Using strong lines diagnostics and the most recent JWST calibrations, we derive extremely-low oxygen abundances ranging from 12+log(O/H)=6.7 to 7.8. By combining this sample with more massive galaxies at similar redshifts, we derive a best-fit relation of 12+{\rm log(O/H)}=$0.39_{-0.02}^{+0.02} \times$ log(\mstar) $+ 4.52_{-0.17}^{+0.17}$, which is steeper than determinations at $z \sim 3$. Our results show a clear redshift evolution in the overall normalization of the relation, galaxies at higher redshift having significantly lower metallicities at a given mass. A comparison with theoretical models provides important constraints on which physical processes, such as metal mixing, star formation or feedback recipes, are important in reproducing the observations. Additionally, these galaxies exhibit star formation rates that are higher by a factor of a few to tens compared to extrapolated relations at similar redshifts or theoretical predictions of main-sequence galaxies, pointing to a recent burst of star formation. All these observations are indicative of highly stochastic star formation and ISM enrichment, expected in these low-mass systems, suggesting that feedback mechanisms in high-$z$ dwarf galaxies might be different from those in place at higher masses., Comment: Submitted to ApJL

We introduce Falcon2-11B, a foundation model trained on over five trillion tokens, and its multimodal counterpart, Falcon2-11B-vlm, which is a vision-to-text model. We report our findings during the training of the Falcon2-11B which follows a multi-stage approach where the early stages are distinguished by their context length and a final stage where we use a curated, high-quality dataset. Additionally, we report the effect of doubling the batch size mid-training and how training loss spikes are affected by the learning rate. The downstream performance of the foundation model is evaluated on established benchmarks, including multilingual and code datasets. The foundation model shows strong generalization across all the tasks which makes it suitable for downstream finetuning use cases. For the vision language model, we report the performance on several benchmarks and show that our model achieves a higher average score compared to open-source models of similar size. The model weights and code of both Falcon2-11B and Falcon2-11B-vlm are made available under a permissive license.

Large language models (LLMs) have recently seen widespread adoption, in both academia and industry. As these models grow, they become valuable intellectual property (IP), reflecting enormous investments by their owners. Moreover, the high cost of cloud-based deployment has driven interest towards deployment to edge devices, yet this risks exposing valuable parameters to theft and unauthorized use. Current methods to protect models' IP on the edge have limitations in terms of practicality, loss in accuracy, or suitability to requirements. In this paper, we introduce a novel hybrid inference algorithm, named SLIP, designed to protect edge-deployed models from theft. SLIP is the first hybrid protocol that is both practical for real-world applications and provably secure, while having zero accuracy degradation and minimal impact on latency. It involves partitioning the model between two computing resources, one secure but expensive, and another cost-effective but vulnerable. This is achieved through matrix decomposition, ensuring that the secure resource retains a maximally sensitive portion of the model's IP while performing a minimal amount of computations, and vice versa for the vulnerable resource. Importantly, the protocol includes security guarantees that prevent attackers from exploiting the partition to infer the secured information. Finally, we present experimental results that show the robustness and effectiveness of our method, positioning it as a compelling solution for protecting LLMs.

For the purpose of causal inference we employ a stochastic model of the data generating process, utilizing individual propensity probabilities for the treatment, and also individual and counterfactual prognosis probabilities for the outcome. We assume a generalized version of the stable unit treatment value assumption, but we do not assume any version of strongly ignorable treatment assignment. Instead of conducting a sensitivity analysis, we utilize the principle of maximum entropy to estimate the distribution of causal effects. We develop a principled middle-way between extreme explanations of the observed data: we do not conclude that an observed association is wholly spurious, and we do not conclude that it is wholly causal. Rather, our conclusions are tempered and we conclude that the association is part spurious and part causal. In an example application we apply our methodology to analyze an observed association between marijuana use and hard drug use.

One of the most promising applications of machine learning (ML) in computational physics is to accelerate the solution of partial differential equations (PDEs). The key objective of ML-based PDE solvers is to output a sufficiently accurate solution faster than standard numerical methods, which are used as a baseline comparison. We first perform a systematic review of the ML-for-PDE solving literature. Of articles that use ML to solve a fluid-related PDE and claim to outperform a standard numerical method, we determine that 79% (60/76) compare to a weak baseline. Second, we find evidence that reporting biases, especially outcome reporting bias and publication bias, are widespread. We conclude that ML-for-PDE solving research is overoptimistic: weak baselines lead to overly positive results, while reporting biases lead to underreporting of negative results. To a large extent, these issues appear to be caused by factors similar to those of past reproducibility crises: researcher degrees of freedom and a bias towards positive results. We call for bottom-up cultural changes to minimize biased reporting as well as top-down structural reforms intended to reduce perverse incentives for doing so.

Domain Generalization techniques aim to enhance model robustness by simulating novel data distributions during training, typically through various augmentation or stylization strategies. However, these methods frequently suffer from limited control over the diversity of generated images and lack assurance that these images span distinct distributions. To address these challenges, we propose FDS, Feedback-guided Domain Synthesis, a novel strategy that employs diffusion models to synthesize novel, pseudo-domains by training a single model on all source domains and performing domain mixing based on learned features. By incorporating images that pose classification challenges to models trained on original samples, alongside the original dataset, we ensure the generation of a training set that spans a broad distribution spectrum. Our comprehensive evaluations demonstrate that this methodology sets new benchmarks in domain generalization performance across a range of challenging datasets, effectively managing diverse types of domain shifts. The implementation is available at: \url{https://github.com/Mehrdad-Noori/FDS.git}.

Large language models (LLMs) are useful tools with the capacity for performing specific types of knowledge work at an effective scale. However, LLM deployments in high-risk and safety-critical domains pose unique challenges, notably the issue of ``hallucination,'' where LLMs can generate fabricated information. This is particularly concerning in settings such as drug safety, where inaccuracies could lead to patient harm. To mitigate these risks, we have developed and demonstrated a proof of concept suite of guardrails specifically designed to mitigate certain types of hallucinations and errors for drug safety, and potentially applicable to other medical safety-critical contexts. These guardrails include mechanisms to detect anomalous documents to prevent the ingestion of inappropriate data, identify incorrect drug names or adverse event terms, and convey uncertainty in generated content. We integrated these guardrails with an LLM fine-tuned for a text-to-text task, which involves converting both structured and unstructured data within adverse event reports into natural language. This method was applied to translate individual case safety reports, demonstrating effective application in a pharmacovigilance processing task. Our guardrail framework offers a set of tools with broad applicability across various domains, ensuring LLMs can be safely used in high-risk situations by eliminating the occurrence of key errors, including the generation of incorrect pharmacovigilance-related terms, thus adhering to stringent regulatory and quality standards in medical safety-critical environments., Comment: 27 pages, 6 figures, 4 tables and supplementary material provided

Preference optimization methods have been successfully applied to improve not only the alignment of large language models (LLMs) with human values, but also specific natural language tasks such as summarization and stylistic continuations. This paper proposes using preference optimization methods on Chain-of-Thought steps in order to improve the reasoning performances of language models. While the chosen answers are obtained from datasets that include reasoning traces, we propose two complementary schemes for generating rejected answers: digit corruption, and weak LLM prompting. Our approach leads to increased accuracy on the GSM8K, AQuA-RAT, and ARC benchmarks for Falcon2-11B and Mistral-7B. For example, the approach can lead to up to a relative 8.47% increase in accuracy on the GSM8K benchmark without any extra annotations. This work suggests that spending resources on creating more datasets of reasoning traces would further boost LLM performances on informal reasoning tasks.

Vision-Language Models (VLMs) such as CLIP have yielded unprecedented performance for zero-shot image classification, yet their generalization capability may still be seriously challenged when confronted to domain shifts. In response, we present Weight Average Test-Time Adaptation (WATT) of CLIP, a pioneering approach facilitating full test-time adaptation (TTA) of this VLM. Our method employs a diverse set of templates for text prompts, augmenting the existing framework of CLIP. Predictions are utilized as pseudo labels for model updates, followed by weight averaging to consolidate the learned information globally. Furthermore, we introduce a text ensemble strategy, enhancing overall test performance by aggregating diverse textual cues. Our findings underscore the efficacy of WATT in enhancing performance across diverse datasets, including CIFAR-10-C, CIFAR-10.1, CIFAR-100-C, VisDA-C, and several other challenging datasets, effectively covering a wide range of domain shifts. Notably, these enhancements are achieved without necessitating additional model transformations or trainable modules. Moreover, compared to other Test-Time Adaptation methods, our approach can operate effectively with just a single image. Highlighting the potential of innovative test-time strategies, this research emphasizes their role in fortifying the adaptability of VLMs. The implementation is available at: \url{https://github.com/Mehrdad-Noori/WATT.git}.

Iron nanoparticles are among the most promising low-dimensional materials in terms of applications. This particularity is attributable to the magnetic properties of these nanoparticles, which exhibit different allotropes as a function of temperature. In this work, we sought to characterise at the atomic scale how their structural and magnetic transformations can be affected by the size. To achieve this objective, we developed a tight-binding model incorporating a magnetic contribution via a Stoner term implemented in a Monte Carlo code to relax the structure and the magnetic state. Using our approach, we show that magnetism is strongly reinforced by the surface, which leads to increase the Curie temperature as the size of the particle decreases contrary to the solid-solid transition temperature. Our work thus provides a deep understanding at the atomic scale of the key factors that determines the structural and magnetic properties of Fe nanoparticles, shedding more light on their unique character which is crucial for further applications.

Urban Air Mobility (UAM) is the envisioned future of inter-city aerial transportation. This paper presents a novel, in-flight connectivity link allocation method for UAM, which dynamically switches between terrestrial cellular and Low Earth Orbit (LEO) satellite networks based on real-time conditions. Our approach prefers cellular networks for cost efficiency, switching to LEO satellites under poor cellular conditions to ensure continuous UAM connectivity. By integrating real-time metrics like signal strength, network congestion, and flight trajectory into the selection process, our algorithm effectively balances cost, minimum data rate requirements, and continuity of communication. Numerical results validate minimization of data-loss while ensuring an optimal selection from the set of available above-threshold data rates at every time sample. Furthermore, insights derived from our study emphasize the importance of hybrid connectivity solutions in ensuring seamless, uninterrupted communication for future urban aerial vehicles., Comment: 6 pages, 6 figures, conference

A deep learning (DL) model, based on a transformer architecture, is trained on a climate-model dataset and compared with a standard linear inverse model (LIM) in the tropical Pacific. We show that the DL model produces more accurate forecasts compared to the LIM when tested on a reanalysis dataset. We then assess the ability of an ensemble Kalman filter to reconstruct the monthly-averaged upper ocean from a noisy set of 24 sea-surface temperature observations designed to mimic existing coral proxy measurements, and compare results for the DL model and LIM. Due to signal damping in the DL model, we implement a novel inflation technique by adding noise from hindcast experiments. Results show that assimilating observations with the DL model yields better reconstructions than the LIM for observation averaging times ranging from one month to one year. The improved reconstruction is due to the enhanced predictive capabilities of the DL model, which map the memory of past observations to future assimilation times.

The traditional method for estimating weather forecast sensitivity to initial conditions uses adjoint models, which are limited to short lead times due to linearization around a control forecast. The advent of deep-learning frameworks enables a new approach using backpropagation and gradient descent to iteratively optimize initial conditions to minimize forecast errors. We apply this approach to forecasts of the June 2021 Pacific Northwest heatwave using the GraphCast model, yielding over 90% reduction in 10-day forecast errors over the Pacific Northwest. Similar improvements are found for Pangu-Weather model forecasts initialized with the GraphCast-derived optimal, suggesting that model error is not an important part of the initial perturbations. Eliminating small scales from the initial perturbations also yields similar forecast improvements. Extending the length of the optimization window, we find forecast improvement to about 23 days, suggesting atmospheric predictability at the upper end of recent estimates., Comment: 11 pages, 4 figures

Data-driven Artificial Intelligence (AI) systems trained using Machine Learning (ML) are shaping an ever-increasing (in size and importance) portion of our lives, including, but not limited to, recommendation systems, autonomous driving technologies, healthcare diagnostics, financial services, and personalized marketing. On the one hand, the outsized influence of these systems imposes a high standard of quality, particularly in the data used to train them. On the other hand, establishing and maintaining standards of Data Quality (DQ) becomes more challenging due to the proliferation of Edge Computing and Internet of Things devices, along with their increasing adoption for training and deploying ML models. The nature of the edge environment -- characterized by limited resources, decentralized data storage, and processing -- exacerbates data-related issues, making them more frequent, severe, and difficult to detect and mitigate. From these observations, it follows that DQ research for edge ML is a critical and urgent exploration track for the safety and robust usefulness of present and future AI systems. Despite this fact, DQ research for edge ML is still in its infancy. The literature on this subject remains fragmented and scattered across different research communities, with no comprehensive survey to date. Hence, this paper aims to fill this gap by providing a global view of the existing literature from multiple disciplines that can be grouped under the umbrella of DQ for edge ML. Specifically, we present a tentative definition of data quality in Edge computing, which we use to establish a set of DQ dimensions. We explore each dimension in detail, including existing solutions for mitigation., Comment: 31 pages, 5 figures

Recommender systems are a kind of data filtering that guides the user to interesting and valuable resources within an extensive dataset. by providing suggestions of products that are expected to match their preferences. However, due to data overloading, recommender systems struggle to handle large volumes of data reliably and accurately before offering suggestions. The main purpose of this work is to address the recommender system's data sparsity and accuracy problems by using the matrix factorization algorithm of collaborative filtering based on the dimensional reduction method and, more precisely, the Nonnegative Matrix Factorization (NMF) combined with ontology. We tested the method and compared the results to other classic methods. The findings showed that the implemented approach efficiently reduces the sparsity of CF suggestions, improves their accuracy, and gives more relevant items as recommendations.

With stunning clarity, JWST has revealed the Universe's first billion years. The scientific community is analyzing a wealth of JWST imaging and spectroscopic data from that era, and is in the process of rewriting the astronomy textbooks. Here, 1.5 years into the JWST science mission, we provide a snapshot of the great progress made towards understanding the initial chapters of our cosmic history. We highlight discoveries and breakthroughs, topics and issues that are not yet understood, and questions that will be addressed in the coming years, as JWST continues its revolutionary observations of the Early Universe. While this compendium is written by a small number of authors, invited to ISSI Bern in March 2024 as part of the 2024 ISSI Breakthrough Workshop, we acknowledge the work of a large community that is advancing our collective understanding of the evolution of the Early Universe., Comment: review article written by the attendees of the 2024 ISSI breakthrough workshop "The first billion year of the Universe", submitted. Comments welcome

Cardiovascular disease is one of the chronic diseases that is on the rise. The complications occur when cardiovascular disease is not discovered early and correctly diagnosed at the right time. Various machine learning approaches, including ontology-based Machine Learning techniques, have lately played an essential role in medical science by building an automated system that can identify heart illness. This paper compares and reviews the most prominent machine learning algorithms, as well as ontology-based Machine Learning classification. Random Forest, Logistic regression, Decision Tree, Naive Bayes, k-Nearest Neighbours, Artificial Neural Network, and Support Vector Machine were among the classification methods explored. The dataset used consists of 70000 instances and can be downloaded from the Kaggle website. The findings are assessed using performance measures generated from the confusion matrix, such as F-Measure, Accuracy, Recall, and Precision. The results showed that the ontology outperformed all the machine learning algorithms.

This work presents an extensive and detailed study on Audio-Visual Speech Recognition (AVSR) for five widely spoken languages: Chinese, Spanish, English, Arabic, and French. We have collected large-scale datasets for each language except for English, and have engaged in the training of supervised learning models. Our model, ViSpeR, is trained in a multi-lingual setting, resulting in competitive performance on newly established benchmarks for each language. The datasets and models are released to the community with an aim to serve as a foundation for triggering and feeding further research work and exploration on Audio-Visual Speech Recognition, an increasingly important area of research. Code available at \href{https://github.com/YasserdahouML/visper}{https://github.com/YasserdahouML/visper}.

Augmented reality (AR) using camera images in mobile devices is becoming popular for tourism promotion. However, obstructions such as tourists appearing in the camera images may cause the camera pose estimation error, resulting in CG misalignment and reduced visibility of the contents. To avoid this problem, Indirect AR (IAR), which does not use real-time camera images, has been proposed. In this method, an omnidirectional image is captured and virtual objects are synthesized on the image in advance. Users can experience AR by viewing a scene extracted from the synthesized omnidirectional image according to the device's sensor. This enables robustness and high visibility. However, if the weather conditions and season in the pre-captured 360 images differs from the current weather conditions and season when AR is experienced, the realism of the AR experience is reduced. To overcome the problem, we propose a method for correcting the intensity and texture of a past omnidirectional image using camera images from mobile devices. We first perform semantic segmentation. We then reproduce the current sky pattern by panoramic image composition and inpainting. For the other areas, we correct the intensity by histogram matching. In experiments, we show the effectiveness of the proposed method using various scenes., Comment: International Workshop on Frontiers of Computer Vision (IW-FCV2024)

The 3D object detection capabilities in urban environments have been enormously improved by recent developments in Light Detection and Range (LiDAR) technology. This paper presents a novel framework that transforms the detection and analysis of 3D objects in traffic scenarios by utilizing the power of elevated LiDAR sensors. We are presenting our methodology's remarkable capacity to collect complex 3D point cloud data, which allows us to accurately and in detail capture the dynamics of urban traffic. Due to the limitation in obtaining real-world traffic datasets, we utilize the simulator to generate 3D point cloud for specific scenarios. To support our experimental analysis, we firstly simulate various 3D point cloud traffic-related objects. Then, we use this dataset as a basis for training and evaluating our 3D object detection models, in identifying and monitoring both vehicles and pedestrians in simulated urban traffic environments. Next, we fine tune the Point Voxel-Region-based Convolutional Neural Network (PV-RCNN) architecture, making it more suited to handle and understand the massive volumes of point cloud data generated by our urban traffic simulations. Our results show the effectiveness of the proposed solution in accurately detecting objects in traffic scenes and highlight the role of LiDAR in improving urban safety and advancing intelligent transportation systems.

The Super-Kamiokande and T2K collaborations present a joint measurement of neutrino oscillation parameters from their atmospheric and beam neutrino data. It uses a common interaction model for events overlapping in neutrino energy and correlated detector systematic uncertainties between the two datasets, which are found to be compatible. Using 3244.4 days of atmospheric data and a beam exposure of $19.7(16.3) \times 10^{20}$ protons on target in (anti)neutrino mode, the analysis finds a 1.9$\sigma$ exclusion of CP-conservation (defined as $J_{CP}=0$) and a preference for the normal mass ordering., Comment: 12 pages, 4 figures

We introduce a pioneering approach to self-supervised learning for point clouds, employing a geometrically informed mask selection strategy called GeoMask3D (GM3D) to boost the efficiency of Masked Auto Encoders (MAE). Unlike the conventional method of random masking, our technique utilizes a teacher-student model to focus on intricate areas within the data, guiding the model's focus toward regions with higher geometric complexity. This strategy is grounded in the hypothesis that concentrating on harder patches yields a more robust feature representation, as evidenced by the improved performance on downstream tasks. Our method also presents a complete-to-partial feature-level knowledge distillation technique designed to guide the prediction of geometric complexity utilizing a comprehensive context from feature-level information. Extensive experiments confirm our method's superiority over State-Of-The-Art (SOTA) baselines, demonstrating marked improvements in classification, and few-shot tasks.

The complex challenge of detecting sarcasm in Arabic speech on social media is increased by the language diversity and the nature of sarcastic expressions. There is a significant gap in the capability of existing models to effectively interpret sarcasm in Arabic, which mandates the necessity for more sophisticated and precise detection methods. In this paper, we investigate the impact of a fundamental preprocessing component on sarcasm speech detection. While emojis play a crucial role in mitigating the absence effect of body language and facial expressions in modern communication, their impact on automated text analysis, particularly in sarcasm detection, remains underexplored. We investigate the impact of emoji exclusion from datasets on the performance of sarcasm detection models in social media content for Arabic as a vocabulary-super rich language. This investigation includes the adaptation and enhancement of AraBERT pre-training models, specifically by excluding emojis, to improve sarcasm detection capabilities. We use AraBERT pre-training to refine the specified models, demonstrating that the removal of emojis can significantly boost the accuracy of sarcasm detection. This approach facilitates a more refined interpretation of language, eliminating the potential confusion introduced by non-textual elements. The evaluated AraBERT models, through the focused strategy of emoji removal, adeptly navigate the complexities of Arabic sarcasm. This study establishes new benchmarks in Arabic natural language processing and presents valuable insights for social media platforms.

Pre-trained vision-language models (VLMs), exemplified by CLIP, demonstrate remarkable adaptability across zero-shot classification tasks without additional training. However, their performance diminishes in the presence of domain shifts. In this study, we introduce CLIP Adaptation duRing Test-Time (CLIPArTT), a fully test-time adaptation (TTA) approach for CLIP, which involves automatic text prompts construction during inference for their use as text supervision. Our method employs a unique, minimally invasive text prompt tuning process, wherein multiple predicted classes are aggregated into a single new text prompt, used as pseudo label to re-classify inputs in a transductive manner. Additionally, we pioneer the standardization of TTA benchmarks (e.g., TENT) in the realm of VLMs. Our findings demonstrate that, without requiring additional transformations nor new trainable modules, CLIPArTT enhances performance dynamically across non-corrupted datasets such as CIFAR-10, corrupted datasets like CIFAR-10-C and CIFAR-10.1, alongside synthetic datasets such as VisDA-C. This research underscores the potential for improving VLMs' adaptability through novel test-time strategies, offering insights for robust performance across varied datasets and environments. The code can be found at: https://github.com/dosowiechi/CLIPArTT.git

The growing capabilities of AI raise questions about their trustworthiness in healthcare, particularly due to opaque decision-making and limited data availability. This paper proposes a novel approach to address these challenges, introducing a Bayesian Monte Carlo Dropout model with kernel modelling. Our model is designed to enhance reliability on small medical datasets, a crucial barrier to the wider adoption of AI in healthcare. This model leverages existing language models for improved effectiveness and seamlessly integrates with current workflows. We demonstrate significant improvements in reliability, even with limited data, offering a promising step towards building trust in AI-driven medical predictions and unlocking its potential to improve patient care.