Search

The success of deep learning in supervised fine-grained recognition for domain-specific tasks relies heavily on expert annotations. The Open-Set for fine-grained Self-Supervised Learning (SSL) problem aims to enhance performance on downstream tasks by strategically sampling a subset of images (the Core-Set) from a large pool of unlabeled data (the Open-Set). In this paper, we propose a novel method, BloomCoreset, that significantly reduces sampling time from Open-Set while preserving the quality of samples in the coreset. To achieve this, we utilize Bloom filters as an innovative hashing mechanism to store both low- and high-level features of the fine-grained dataset, as captured by Open-CLIP, in a space-efficient manner that enables rapid retrieval of the coreset from the Open-Set. To show the effectiveness of the sampled coreset, we integrate the proposed method into the state-of-the-art fine-grained SSL framework, SimCore [1]. The proposed algorithm drastically outperforms the sampling strategy of the baseline in SimCore [1] with a $98.5\%$ reduction in sampling time with a mere $0.83\%$ average trade-off in accuracy calculated across $11$ downstream datasets., Comment: Accepted at ICASSP 2025

We study in detail the production of primordial black holes (PBHs), as well as their mass and initial spin, due to the phase transitions corresponding to radiative symmetry breaking (RSB) and featuring a large supercooling. The latter property allows us to use a model-independent approach. In this context, we demonstrate that the decay rate of the false vacuum grows exponentially with time to a high degree of accuracy, justifying a time dependence commonly assumed in the literature. Our study provides ready-to-use results for determining the abundance, mass and initial spin of PBHs generated in a generic RSB model with large supercooling. We find that PBHs are generically produced in a broad region of the model-independent parameter space. Notably, we identify the subregion that may explain recently observed microlensing anomalies. Additionally, we show that a simple Standard-Model extension, with right-handed neutrinos and gauged $B-L$ featuring RSB, may explain an anomaly of this sort in a region of its parameter space., Comment: 29 pages, 10 figures

The rapid development of large Vision-Language Models (VLMs) has led to impressive results on academic benchmarks, primarily in widely spoken languages. However, significant gaps remain in the ability of current VLMs to handle low-resource languages and varied cultural contexts, largely due to a lack of high-quality, diverse, and safety-vetted data. Consequently, these models often struggle to understand low-resource languages and cultural nuances in a manner free from toxicity. To address these limitations, we introduce Maya, an open-source Multimodal Multilingual model. Our contributions are threefold: 1) a multilingual image-text pretraining dataset in eight languages, based on the LLaVA pretraining dataset; 2) a thorough analysis of toxicity within the LLaVA dataset, followed by the creation of a novel toxicity-free version across eight languages; and 3) a multilingual image-text model supporting these languages, enhancing cultural and linguistic comprehension in vision-language tasks. Code available at https://github.com/nahidalam/maya.

In this paper, we have proposed a novel route for the realisation of persistent spin texture (PST). We have shown from symmetry considerations that in non-polar chiral systems, bands with specific orbital characters around a high symmetry point with $D_{2}$ little group may admit a single spin dependent term in the low energy $\bf{k.p}$ model Hamiltonian that naturally leads to PST. Considering a $2D$ plane in the Brillouin zone (BZ), we have further argued that in such chiral systems the PST is transpired due to the comparable strengths of the Dresselhaus and Weyl (radial) interaction parameters where the presence of these two terms are allowed by the $D{_2}$ symmetry. Finally using first principles density functional theory (DFT) calculations we have identified that the non-polar chiral compounds Y$_3$TaO$_7$ and AsBr$_3$ displays PST for the conduction band and valence band respectively around the $\Gamma$ point having $D{_2}$ little group and predominantly Ta-$d_{xz}$ orbital character for Y$_3$TaO$_7$ and Br-$p{_x}$ orbital character for AsBr$_3$ corroborating our general strategy. Our results for the realisation of PST in non-polar chiral systems thereby broaden the class of materials displaying PST that can be employed for application in spin-orbitronics.

Background: Explainability in phishing detection model can support a further solution of phishing attack mitigation by increasing trust and understanding how phishing can be detected. Objective: The aims of this study to determine and best recommendation to apply an approach which has several components with abilities to fulfil the critical needs Methods: A methodology starting with analyzing both black-box and white-box models to get the pros and cons specifically in phishing detection. The conclusion of the analysis will be validated by experiment using a set of well-known algorithms and public phishing datasets. Experimental metrics covers 3 measurements such as predictive accuracy and explainability metrics. Conclusion: Both models are comparable in terms of interpretability and consistency, with room for improvement in diverse datasets. EBM as an example of white-box model is generally better suited for applications requiring explainability and actionable insights. Finally, each model, white-box and black-box model has positive and negative aspects both for performance metric and for explainable metric. It is important to consider the objective of model usage.

We introduce a general-purpose, human-in-the-loop dual dialogue system to support mental health care professionals. The system, co-designed with care providers, is conceptualized to assist them in interacting with care seekers rather than functioning as a fully automated dialogue system solution. The AI assistant within the system reduces the cognitive load of mental health care providers by proposing responses, analyzing conversations to extract pertinent themes, summarizing dialogues, and recommending localized relevant content and internet-based cognitive behavioral therapy exercises. These functionalities are achieved through a multi-agent system design, where each specialized, supportive agent is characterized by a large language model. In evaluating the multi-agent system, we focused specifically on the proposal of responses to emotionally distressed care seekers. We found that the proposed responses matched a reasonable human quality in demonstrating empathy, showing its appropriateness for augmenting the work of mental health care providers., Comment: Update: Render figures properly and update title

Phishing attacks remain a persistent threat to online security, demanding robust detection methods. This study investigates the use of machine learning to identify phishing URLs, emphasizing the crucial role of feature selection and model interpretability for improved performance. Employing Recursive Feature Elimination, the research pinpointed key features like "length_url," "time_domain_activation" and "Page_rank" as strong indicators of phishing attempts. The study evaluated various algorithms, including CatBoost, XGBoost, and Explainable Boosting Machine, assessing their robustness and scalability. XGBoost emerged as highly efficient in terms of runtime, making it well-suited for large datasets. CatBoost, on the other hand, demonstrated resilience by maintaining high accuracy even with reduced features. To enhance transparency and trustworthiness, Explainable AI techniques, such as SHAP, were employed to provide insights into feature importance. The study's findings highlight that effective feature selection and model interpretability can significantly bolster phishing detection systems, paving the way for more efficient and adaptable defenses against evolving cyber threats

We present MetaMetrics-MT, an innovative metric designed to evaluate machine translation (MT) tasks by aligning closely with human preferences through Bayesian optimization with Gaussian Processes. MetaMetrics-MT enhances existing MT metrics by optimizing their correlation with human judgments. Our experiments on the WMT24 metric shared task dataset demonstrate that MetaMetrics-MT outperforms all existing baselines, setting a new benchmark for state-of-the-art performance in the reference-based setting. Furthermore, it achieves comparable results to leading metrics in the reference-free setting, offering greater efficiency., Comment: Preprint

Code-switching, the phenomenon of alternating between two or more languages in a single conversation, presents unique challenges for Natural Language Processing (NLP). Most existing research focuses on either syntactic constraints or neural generation, with few efforts to integrate linguistic theory with large language models (LLMs) for generating natural code-switched text. In this paper, we introduce EZSwitch, a novel framework that combines Equivalence Constraint Theory (ECT) with LLMs to produce linguistically valid and fluent code-switched text. We evaluate our method using both human judgments and automatic metrics, demonstrating a significant improvement in the quality of generated code-switching sentences compared to baseline LLMs. To address the lack of suitable evaluation metrics, we conduct a comprehensive correlation study of various automatic metrics against human scores, revealing that current metrics often fail to capture the nuanced fluency of code-switched text. Additionally, we create CSPref, a human preference dataset based on human ratings and analyze model performance across ``hard`` and ``easy`` examples. Our findings indicate that incorporating linguistic constraints into LLMs leads to more robust and human-aligned generation, paving the way for scalable code-switching text generation across diverse language pairs.

In recent years, personalized diffusion-based text-to-image generative tasks have been a hot topic in computer vision studies. A robust diffusion model is determined by its ability to perform near-perfect reconstruction of certain product outcomes given few related input samples. Unfortunately, the current prominent diffusion-based finetuning technique falls short in maintaining the foreground object consistency while being constrained to produce diverse backgrounds in the image outcome. In the worst scenario, the overfitting issue may occur, meaning that the foreground object is less controllable due to the condition above, for example, the input prompt information is transferred ambiguously to both foreground and background regions, instead of the supposed background region only. To tackle the issues above, we proposed Hypnos, a highly precise foreground-focused diffusion finetuning technique. On the image level, this strategy works best for inanimate object generation tasks, and to do so, Hypnos implements two main approaches, namely: (i) a content-centric prompting strategy and (ii) the utilization of our additional foreground-focused discriminative module. The utilized module is connected with the diffusion model and finetuned with our proposed set of supervision mechanism. Combining the strategies above yielded to the foreground-background disentanglement capability of the diffusion model. Our experimental results showed that the proposed strategy gave a more robust performance and visually pleasing results compared to the former technique. For better elaborations, we also provided extensive studies to assess the fruitful outcomes above, which reveal how personalization behaves in regard to several training conditions., Comment: 26 pages, 12 figures, to appear on the Rich Media with Generative AI workshop in conjunction with Asian Conference on Computer Vision (ACCV) 2024

Large-scale molecular representation methods have revolutionized applications in material science, such as drug discovery, chemical modeling, and material design. With the rise of transformers, models now learn representations directly from molecular structures. In this study, we develop an encoder-decoder model based on BART that is capable of leaning molecular representations and generate new molecules. Trained on SELFIES, a robust molecular string representation, our model outperforms existing baselines in downstream tasks, demonstrating its potential in efficient and effective molecular data analysis and manipulation., Comment: NeurIPS AI4Mat 2024

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data., Comment: Preprint

Machine learning (ML) course for undergraduates face challenges in assessing student learning and providing practical exposure. Group project-based learning, an increasingly popular form of experiential learning in CS education, encounters certain limitation in participation and non-participation from a few students. Studies also suggest that students find longer programming assignments and project-based assessments distracting and struggle to maintain focus when they coincide with other courses. To tackle these issues, we introduced FlashHack: a monitored, incremental, in-classroom micro Hackathon that combines project-based learning with Hackathon elements. Engaging 229 third year CS undergraduate students in teams of four, FlashHack prompted them to tackle predefined challenges using machine learning techniques within a set timeframe. Assessment criteria emphasized machine learning application, problem-solving, collaboration, and creativity. Our results indicate high student engagement and satisfaction, alongside simplified assessment processes for instructors. This experience report outlines the Hackathon design and implementation, highlights successes and areas for improvement making it feasible for replication by interested computing educators.

Recently, numerous preference optimization algorithms have been introduced as extensions to the Direct Preference Optimization (DPO) family. While these methods have successfully aligned models with human preferences, there is a lack of understanding regarding the contributions of their additional components. Moreover, fair and consistent comparisons are scarce, making it difficult to discern which components genuinely enhance downstream performance. In this work, we propose RainbowPO, a unified framework that demystifies the effectiveness of existing DPO methods by categorizing their key components into seven broad directions. We integrate these components into a single cohesive objective, enhancing the performance of each individual element. Through extensive experiments, we demonstrate that RainbowPO outperforms existing DPO variants. Additionally, we provide insights to guide researchers in developing new DPO methods and assist practitioners in their implementations.

Understanding the quality of a performance evaluation metric is crucial for ensuring that model outputs align with human preferences. However, it remains unclear how well each metric captures the diverse aspects of these preferences, as metrics often excel in one particular area but not across all dimensions. To address this, it is essential to systematically calibrate metrics to specific aspects of human preference, catering to the unique characteristics of each aspect. We introduce MetaMetrics, a calibrated meta-metric designed to evaluate generation tasks across different modalities in a supervised manner. MetaMetrics optimizes the combination of existing metrics to enhance their alignment with human preferences. Our metric demonstrates flexibility and effectiveness in both language and vision downstream tasks, showing significant benefits across various multilingual and multi-domain scenarios. MetaMetrics aligns closely with human preferences and is highly extendable and easily integrable into any application. This makes MetaMetrics a powerful tool for improving the evaluation of generation tasks, ensuring that metrics are more representative of human judgment across diverse contexts., Comment: Preprint

Natural Language Explanation (NLE) aims to elucidate the decision-making process by providing detailed, human-friendly explanations in natural language. It helps demystify the decision-making processes of large vision-language models (LVLMs) through the use of language models. While existing methods for creating a Vision Question-Answering with Natural Language Explanation (VQA-NLE) datasets can provide explanations, they heavily rely on human annotations that are time-consuming and costly. In this study, we propose a novel approach that leverages LVLMs to efficiently generate high-quality synthetic VQA-NLE datasets. By evaluating our synthetic data, we showcase how advanced prompting techniques can lead to the production of high-quality VQA-NLE data. Our findings indicate that this proposed method achieves up to 20x faster than human annotation, with only a minimal decrease in qualitative metrics, achieving robust quality that is nearly equivalent to human-annotated data. Furthermore, we show that incorporating visual prompts significantly enhances the relevance of text generation. Our study paves the way for a more efficient and robust automated generation of multi-modal NLE data, offering a promising solution to the problem., Comment: Preprint

Preference tuning is a crucial process for aligning deep generative models with human preferences. This survey offers a thorough overview of recent advancements in preference tuning and the integration of human feedback. The paper is organized into three main sections: 1) introduction and preliminaries: an introduction to reinforcement learning frameworks, preference tuning tasks, models, and datasets across various modalities: language, speech, and vision, as well as different policy approaches, 2) in-depth exploration of each preference tuning approach: a detailed analysis of the methods used in preference tuning, and 3) applications, discussion, and future directions: an exploration of the applications of preference tuning in downstream tasks, including evaluation methods for different modalities, and an outlook on future research directions. Our objective is to present the latest methodologies in preference tuning and model alignment, enhancing the understanding of this field for researchers and practitioners. We hope to encourage further engagement and innovation in this area., Comment: Survey paper

We perform a study to obtain the initial spin or the nondimensional Kerr parameter $a_{*}$ of primordial black holes (PBHs) created during the radiation dominated phase of the universe from not only nearly monochromatic but also broad curvature power spectra. Motivated by inflation and first-order phase transitions, we consider a power law shape for the curvature perturbation. Although we can naturally neglect the contribution from the length scales smaller than the scale of interest, that from the larger scales may potentially be significant for a broad power spectrum, for which the spin is sensitive to the width of the power spectrum. So, we introduce a width parameter $r_{k}$, the ratio of the largest scale to the length of interest. We find that the root mean square of $a_{*}$ is largest for PBHs created from locally nearly scale invariant curvature power spectra with $r_{k} \sim 3.5$. The upper limit is $\sim 1\times 10^{-4}$ for $M=10^{17}-10^{23}$ g, $\sim 1.7\times 10^{-4}$ for $M=1-100 M_{\odot}$ and $\sim 2.5\times 10^{-3}$ even for an incredibly large mass of $M=10^{50}$ g., Comment: 17 pages, 5 figures

This work addresses the problem of finding a consensus point in the state space ($\mathbb{R}^2$) for a multi-agent system that is comprised of $N$ identical double integrator agents. It is assumed that each agent operates under constrained control input (i.e., $|u_i(t)| \leq 1$ $\forall i = 1, \hdots N$). Further, a fixed fuel budget is also assumed i.e., the total amount of cumulative input that can be expended is limited by $\int_0^{t_f}|u(t)|dt \le \beta$. First, the attainable set $\mathcal{A}(t,x_0,\beta)$ at time $t$, which is the set of all states that an agent can attain starting from initial conditions $x_0$ under the fuel budget constraints at time $t$ is computed for every agent. This attainable set is a convex set for all $t\ge0$. Then the minimum time to consensus is the minimum time $\bar{t}$ at which attainable sets of all agents intersect, and the consensus point is the point of intersection. A closed-form expression for the minimum time consensus point is provided for the case of three agents. Then, using Helly's theorem, the intersection will be non-empty at a time when all the $N \choose 3$ triplets of agents have non-empty intersection. The computation of minimum time consensus for all $N \choose 3$ triplets is performed independently and can be distributed among all the $N$ agents. Finally, the overall minimum time to consensus is given by the triplet that has the highest minimum time to consensus. Further, the intersection of all the attainable sets of this triplet gives the minimum time consensus point for all $N$ agents.

Development of efficient and high-performing electrolytes is crucial for advancing energy storage technologies, particularly in batteries. Predicting the performance of battery electrolytes rely on complex interactions between the individual constituents. Consequently, a strategy that adeptly captures these relationships and forms a robust representation of the formulation is essential for integrating with machine learning models to predict properties accurately. In this paper, we introduce a novel approach leveraging a transformer-based molecular representation model to effectively and efficiently capture the representation of electrolyte formulations. The performance of the proposed approach is evaluated on two battery property prediction tasks and the results show superior performance compared to the state-of-the-art methods., Comment: Accepted in ML4LMS Workshop at ICML 2024

We propose a novel method to study the ultra-light scalars, where compact rotating objects undergo the phenomenon of superradiance to create gravitational waves and neutrino flux signals. The neutrino flux results from the 'right' coupling between the ultra-light scalars and the neutrinos. We study the intertwining of gravitational waves and neutrino flux signals produced from a single source and elaborate if and when the signals can be detected in existing and upcoming experiments in a direct manner. We also discuss an indirect way to test it by means of cosmic neutrino background which can be detected by upcoming PTOLEMY experiment., Comment: 16 pages, 5 figures, 1 table

We conduct an analysis of a $U(1)_{B-L}$ extended inert doublet model and obtained the parameter space allowing strong first order phase transitions. We show that a large part of the parameter space can cause double first-order phase transitions. Whereas both of these phase transitions can generate a detectable stochastic gravitational wave background, one of them can create primordial black holes with appreciable abundance. The primordial black holes generated at the high scale transition can account for the dark matter maintaining the correct relic abundance. We also show specific benchmark cases and their consequences from the aspect of primordial black holes and gravitational waves., Comment: 22 pages, 9 figures, 3 tables; discussions updated, figures and tables modified; matches published version

Southeast Asia (SEA) is a region rich in linguistic diversity and cultural variety, with over 1,300 indigenous languages and a population of 671 million people. However, prevailing AI models suffer from a significant lack of representation of texts, images, and audio datasets from SEA, compromising the quality of AI models for SEA languages. Evaluating models for SEA languages is challenging due to the scarcity of high-quality datasets, compounded by the dominance of English training data, raising concerns about potential cultural misrepresentation. To address these challenges, we introduce SEACrowd, a collaborative initiative that consolidates a comprehensive resource hub that fills the resource gap by providing standardized corpora in nearly 1,000 SEA languages across three modalities. Through our SEACrowd benchmarks, we assess the quality of AI models on 36 indigenous languages across 13 tasks, offering valuable insights into the current AI landscape in SEA. Furthermore, we propose strategies to facilitate greater AI advancements, maximizing potential utility and resource equity for the future of AI in SEA., Comment: https://seacrowd.github.io/ Accepted in EMNLP 2024

Performance prediction is a method to estimate the performance of Language Models (LMs) on various Natural Language Processing (NLP) tasks, mitigating computational costs associated with model capacity and data for fine-tuning. Our paper presents ProxyLM, a scalable task- and language-agnostic framework designed to predict the performance of LMs using proxy models. These proxy models act as surrogates, approximating the performance of the LM of interest. By leveraging these proxy models, ProxyLM significantly reduces computational overhead in task evaluations, achieving up to a 37.08x speedup over traditional methods, even with our smallest proxy models. Our results across multiple multilingual NLP tasks and various robustness tests demonstrate that ProxyLM not only adapts well to previously unseen languages in pre-trained LMs, but also generalizes effectively across different datasets, outperforming the state-of-the-art by at least 1.78x in terms of root-mean-square error (RMSE)., Comment: Preprint

Recent developments in self-supervised learning (SSL) have made it possible to learn data representations without the need for annotations. Inspired by the non-contrastive SSL approach (SimSiam), we introduce a novel framework SIMSAM to compute the Semantic Affinity Matrix, which is significant for unsupervised image segmentation. Given an image, SIMSAM first extracts features using pre-trained DINO-ViT, then projects the features to predict the correlations of dense features in a non-contrastive way. We show applications of the Semantic Affinity Matrix in object segmentation and semantic segmentation tasks. Our code is available at https://github.com/chandagrover/SimSAM., Comment: 6 Pages-Main Paper , 6 figures, 6Tables (Main Paper), ICIP 2024, 8 Pages: Supplementary

Words have been represented in a high-dimensional vector space that encodes their semantic similarities, enabling downstream applications such as retrieving synonyms, antonyms, and relevant contexts. However, despite recent advances in multilingual language models (LMs), the effectiveness of these models' representations in semantic retrieval contexts has not been comprehensively explored. To fill this gap, this paper introduces the MINERS, a benchmark designed to evaluate the ability of multilingual LMs in semantic retrieval tasks, including bitext mining and classification via retrieval-augmented contexts. We create a comprehensive framework to assess the robustness of LMs in retrieving samples across over 200 diverse languages, including extremely low-resource languages in challenging cross-lingual and code-switching settings. Our results demonstrate that by solely retrieving semantically similar embeddings yields performance competitive with state-of-the-art approaches, without requiring any fine-tuning., Comment: Accepted by EMNLP 2024 Findings