Search

We present a simplified derivation of the optimal checkpoint interval in Young_1974 [1]. The optimal checkpoint interval derivation in [1] is based on minimizing the total lost time as an objective-function. Lost time is a function of checkpoint interval, checkpoint save time, and average failure time. This simplified derivation yields lost-time-optimal that is identical to the one derived in [1]. For large scale-out super-computer or datacenter systems, what is important is the selection of optimal checkpoint interval that maximizes availability. We show that availability-optimal checkpoint interval is different from the one derived in [1]. However, availability-optimal checkpoint interval is asymptotically same as lost-time-optimal checkpoint interval for certain conditions on checkpoint save and recovery time. We show that these optimal checkpoint intervals hold in situations where the error detection latency is significantly smaller than any selected checkpoint interval. However, in cases where the error detection latency is very large then the optimal checkpoint interval is greater than or equal to the error detection latency., Comment: 10 pages, 5 figures

The Automated Audio Captioning (AAC) task aims to describe an audio signal using natural language. To evaluate machine-generated captions, the metrics should take into account audio events, acoustic scenes, paralinguistics, signal characteristics, and other audio information. Traditional AAC evaluation relies on natural language generation metrics like ROUGE and BLEU, image captioning metrics such as SPICE and CIDEr, or Sentence-BERT embedding similarity. However, these metrics only compare generated captions to human references, overlooking the audio signal itself. In this work, we propose MACE (Multimodal Audio-Caption Evaluation), a novel metric that integrates both audio and reference captions for comprehensive audio caption evaluation. MACE incorporates audio information from audio as well as predicted and reference captions and weights it with a fluency penalty. Our experiments demonstrate MACE's superior performance in predicting human quality judgments compared to traditional metrics. Specifically, MACE achieves a 3.28% and 4.36% relative accuracy improvement over the FENSE metric on the AudioCaps-Eval and Clotho-Eval datasets respectively. Moreover, it significantly outperforms all the previous metrics on the audio captioning evaluation task. The metric is opensourced at https://github.com/satvik-dixit/mace

Vision Language Models (VLMs) have been successful at many chart comprehension tasks that require attending to both the images of charts and their accompanying textual descriptions. However, it is not well established how VLM performance profiles map to human-like behaviors. If VLMs can be shown to have human-like chart comprehension abilities, they can then be applied to a broader range of tasks, such as designing and evaluating visualizations for human readers. This paper lays the foundations for such applications by evaluating the accuracy of zero-shot prompting of VLMs on graphical perception tasks with established human performance profiles. Our findings reveal that VLMs perform similarly to humans under specific task and style combinations, suggesting that they have the potential to be used for modeling human performance. Additionally, variations to the input stimuli show that VLM accuracy is sensitive to stylistic changes such as fill color and chart contiguity, even when the underlying data and data mappings are the same.

In October 2024, The object BL Lac experienced a brightest flaring event in gamma-ray ($>$100 MeV) with a historical $\gamma$-ray flux of $\sim$10$^{-5}$ erg cm$^{-2}$ s$^{-1}$. Soon after the event was followed across the waveband and in X-ray (0.3-10 keV) it was also found to be flaring with the maximum flux achieved during this event as 8.30$\times$10$^{-11}$ erg cm$^{-2}$ s$^{-1}$. The high gamma-ray significance enables us to probe the shortest time scale variability possible and for that, we produced the orbital binned light curve, 5 minutes binned light curve, and the 2 minutes binned light curve. A clear variation is seen in the 5-minute light curve and is fitted with the sum of exponentials to derive the rise and decay time scale which ranges between 3 to 12 minutes. The fastest variability time is also estimated to be an order of 1 minute from 2 minute. The estimated size of the emission region is very small (10$^{13}$ cm) compared to the size of the black hole event horizon. The location of the emission region is estimated to be very close to the supermassive black hole (10$^{14}$ cm) and much inside the BLR (0.1 pc). We discussed the possible way to explain this fast-flux variability in BL Lac., Comment: Submitted

In this work, we studied the broadband temporal and spectral properties of the flat-spectrum radio quasar (FSRQ) Ton 599 and explored the one-zone leptonic model to fit the broadband spectral energy distribution (SED). We collected the long-term data from June 2020 to August 2024 when the source was in a long flaring episode. We used the Bayesian block methodology to identify the various flux states which included three flares. The broadband fractional variability is estimated during two flaring states and in the total light curve. The Fvar distribution with respect to frequency shows a double hump structure similar to broadband SED. The Power spectral density (PSD) shows a pink-noise kind of stochastic variability in the light curve and we do not see any break in the power spectrum suggesting a much longer characteristic time scale is involved in gamma-ray variability. The flux distribution is well-fitted with a double log-normal flux distribution suggesting the variability of non-linear in nature. The gamma-ray, optical, and X-ray emissions were found to be highly correlated with a zero time lag suggesting a co-spatial origin. We used the one-zone leptonic model to reproduce the broad-band spectrum in the energy range from IR to very high-energy gamma-ray. The increase in the magnetic field and the Doppler factor were found to be the main cause for high flux states. The XMM-Newton spectra taken during one of the flaring durations exhibit a signature of black body emission from the accretion disk suggesting a possible disk-jet coupling. This has also been indicated by the gamma-ray flux distribution which shows the distribution as non-linear in nature mostly seen in galactic X-ray binaries or AGN where emission is dominated by the accretion disk. This possibility of disk-jet coupling will be explored in the coming works., Comment: Submitted

The memory burden effect is an explicit resolution to the information paradox by which an evaporating black hole acquires quantum hair, which then suppresses its rate of mass loss with respect to the semi-classical Hawking rate. We show that this has significant implications for particle dark matter that captures in neutron stars and forms black holes that go on to consume the host star. In particular, we show that constraints on the nucleon scattering cross section and mass of spin-0 and spin-1/2 dark matter would be extended by several orders of magnitude., Comment: 4 pages revtex + references, 1 figure

The rapid evolution of Large Language Models (LLMs) offers promising potential to alleviate the global scarcity of mental health professionals. However, LLMs' alignment with essential mental health counseling competencies remains understudied. We introduce CounselingBench, a novel NCMHCE-based benchmark evaluating LLMs across five key mental health counseling competencies. Testing 22 general-purpose and medical-finetuned LLMs, we find frontier models exceed minimum thresholds but fall short of expert-level performance, with significant variations: they excel in Intake, Assessment & Diagnosis yet struggle with Core Counseling Attributes and Professional Practice & Ethics. Medical LLMs surprisingly underperform generalist models accuracy-wise, while at the same time producing slightly higher-quality justifications but making more context-related errors. Our findings highlight the complexities of developing AI systems for mental health counseling, particularly for competencies requiring empathy and contextual understanding. We found that frontier LLMs perform at a level exceeding the minimal required level of aptitude for all key mental health counseling competencies, but fall short of expert-level performance, and that current medical LLMs do not significantly improve upon generalist models in mental health counseling competencies. This underscores the critical need for specialized, mental health counseling-specific fine-tuned LLMs that rigorously aligns with core competencies combined with appropriate human supervision before any responsible real-world deployment can be considered., Comment: 9 Pages, In Submission to NAACL 2025

In direct searches of dark matter, multi-scatter signatures are now being sought to probe scattering cross sections that are large enough to make the detector optically thick to incident particles. We provide some significant updates to the multi-scatter program. Using considerations of energy deposition, we derive the reaches in cross section and mass of various proposed large-volume detectors: a kilotonne fiducial mass "module of opportunity" at DUNE, a kilotonne xenon detector suggested for neutrinoless double beta decay, the gaseous detector CYGNUS, and the dark matter detectors XLZD and Argo. Where the velocity vector can be reconstructed event-by-event, the Galactic dark matter velocity distribution may be inferred. We exploit this to show that halo substructure such as tidal streams can be picked up if they make up about 10% of the local dark matter density., Comment: 9 pages revtex4 + references, 3 figures, 1 table

Distributed Machine Learning (DML) on resource-constrained edge devices holds immense potential for real-world applications. However, achieving fast convergence in DML in these heterogeneous environments remains a significant challenge. Traditional frameworks like Bulk Synchronous Parallel and Asynchronous Stochastic Parallel rely on frequent, small updates that incur substantial communication overhead and hinder convergence speed. Furthermore, these frameworks often employ static dataset sizes, neglecting the heterogeneity of edge devices and potentially leading to straggler nodes that slow down the entire training process. The straggler nodes, i.e., edge devices that take significantly longer to process their assigned data chunk, hinder the overall training speed. To address these limitations, this paper proposes Hermes, a novel probabilistic framework for efficient DML on edge devices. This framework leverages a dynamic threshold based on recent test loss behavior to identify statistically significant improvements in the model's generalization capability, hence transmitting updates only when major improvements are detected, thereby significantly reducing communication overhead. Additionally, Hermes employs dynamic dataset allocation to optimize resource utilization and prevents performance degradation caused by straggler nodes. Our evaluations on a real-world heterogeneous resource-constrained environment demonstrate that Hermes achieves faster convergence compared to state-of-the-art methods, resulting in a remarkable $13.22$x reduction in training time and a $62.1\%$ decrease in communication overhead., Comment: 11 pages, 19 figures, 3 tables; code: https://github.com/DaSH-Lab-CSIS/Hermes

The proliferation of tobacco-related content on social media platforms poses significant challenges for public health monitoring and intervention. This paper introduces a novel multi-modal deep learning framework named Flow-Attention Adaptive Semantic Hierarchical Fusion (FLAASH) designed to analyze tobacco-related video content comprehensively. FLAASH addresses the complexities of integrating visual and textual information in short-form videos by leveraging a hierarchical fusion mechanism inspired by flow network theory. Our approach incorporates three key innovations, including a flow-attention mechanism that captures nuanced interactions between visual and textual modalities, an adaptive weighting scheme that balances the contribution of different hierarchical levels, and a gating mechanism that selectively emphasizes relevant features. This multi-faceted approach enables FLAASH to effectively process and analyze diverse tobacco-related content, from product showcases to usage scenarios. We evaluate FLAASH on the Multimodal Tobacco Content Analysis Dataset (MTCAD), a large-scale collection of tobacco-related videos from popular social media platforms. Our results demonstrate significant improvements over existing methods, outperforming state-of-the-art approaches in classification accuracy, F1 score, and temporal consistency. The proposed method also shows strong generalization capabilities when tested on standard video question-answering datasets, surpassing current models. This work contributes to the intersection of public health and artificial intelligence, offering an effective tool for analyzing tobacco promotion in digital media., Comment: Under review at International Journal of Computer Vision; 20 pages, 4 figures, 5 tables

Early screening for cancer has proven to improve the survival rate and spare patients from intensive and costly treatments due to late diagnosis. Cancer screening in the healthy population involves an initial risk stratification step to determine the screening method and frequency, primarily to optimize resource allocation by targeting screening towards individuals who draw most benefit. For most screening programs, age and clinical risk factors such as family history are part of the initial risk stratification algorithm. In this paper, we focus on developing a blood marker-based risk stratification approach, which could be used to identify patients with elevated cancer risk to be encouraged for taking a diagnostic test or participate in a screening program. We demonstrate that the combination of simple, widely available blood tests, such as complete blood count and complete metabolic panel, could potentially be used to identify patients at risk for colorectal, liver, and lung cancers with areas under the ROC curve of 0.76, 0.85, 0.78, respectively. Furthermore, we hypothesize that such an approach could not only be used as pre-screening risk assessment for individuals but also as population health management tool, for example to better interrogate the cancer risk in certain sub-populations.

Sharing protected health information (PHI) is critical for furthering biomedical research. Before data can be distributed, practitioners often perform deidentification to remove any PHI contained in the text. Contemporary deidentification methods are evaluated on highly saturated datasets (tools achieve near-perfect accuracy) which may not reflect the full variability or complexity of real-world clinical text and annotating them is resource intensive, which is a barrier to real-world applications. To address this gap, we developed an adversarial approach using a large language model (LLM) to re-identify the patient corresponding to a redacted clinical note and evaluated the performance with a novel De-Identification/Re-Identification (DIRI) method. Our method uses a large language model to reidentify the patient corresponding to a redacted clinical note. We demonstrate our method on medical data from Weill Cornell Medicine anonymized with three deidentification tools: rule-based Philter and two deep-learning-based models, BiLSTM-CRF and ClinicalBERT. Although ClinicalBERT was the most effective, masking all identified PII, our tool still reidentified 9% of clinical notes Our study highlights significant weaknesses in current deidentification technologies while providing a tool for iterative development and improvement.

Optical data center networks (DCNs) are renovating the infrastructure design for the cloud in the post Moore's law era. The fact that optical DCNs rely on optical circuits of microsecond-scale durations makes nanosecond-precision time synchronization essential for the correct functioning of routing on the network fabric. However, current studies on optical DCNs neglect the fundamental need for accurate time synchronization. In this paper, we bridge the gap by developing Nanosecond Optical Synchronization (NOS), the first nanosecond-precision synchronization solution for optical DCNs general to various optical hardware. NOS builds clock propagation trees on top of the dynamically reconfigured circuits in optical DCNs, allowing switches to seek better sync parents throughout time. It predicts drifts in the tree-building process, which enables minimization of sync errors. We also tailor today's sync protocols to the needs of optical DCNs, including reducing the number of sync messages to fit into short circuit durations and correcting timestamp errors for higher sync accuracy. Our implementation on programmable switches shows 28ns sync accuracy in a 192-ToR setting.

Matter falling onto black holes, {also called} accretion discs, emit intense high-energy radiation. Accretion discs during {hard to hard intermediate} spectral states also emit bipolar outflows. Radiation drag was supposed to impose the upper limit on the terminal speed. It was later shown that a radiation field around an advective accretion disc imposes no upper limit on speed, about a few hundred of Schwarzschild radius from the disc surface. We {study radiatively driven electron-proton and electron-positron jets, for gemeotrically thick and slim transonic discs} by using numerical simulation. We show that pair-dominated jets can reach ultra-relativistic speeds by radiation driving. We also discuss at what limits radiative acceleration may fail., Comment: 10 pages, 6 figures. Refereed conference procedings of International Symposium on Recent Developments i Relativistic Astrophysics (ISRA 2023), held in SRM University Sikkim

The rise of Large Language Models (LLMs) has accentuated the need for diverse, high-quality pre-training data. Synthetic data emerges as a viable solution to the challenges of data scarcity and inaccessibility. While previous literature has focused predominantly on the quality and quantity of real data, our work enables the measurement of diversity in synthetic data and explores its impact on LLM performance. We study the downstream effects of synthetic data diversity during both the pre-training and fine-tuning stages by introducing a new diversity metric, \textit{LLM cluster-agent}, designed to evaluate the diversity of synthetic datasets. Through a series of controlled experiments with models of 350M and 1.4B parameters, we demonstrate that the proposed cluster-based LLM scoring of diversity correlates positively with both pre-training and supervised fine-tuning performance. Our findings also reveal that synthetic data diversity in pre-training affects supervised fine-tuning more significantly than pre-training itself, even for smaller models. We hope this study advances our understanding of the optimal use of synthetic data in LLM training and opens new avenues for efficient data generation processes.

In this study, we apply two pillars of Scientific Machine Learning: Neural Ordinary Differential Equations (Neural ODEs) and Universal Differential Equations (UDEs) to the Chandrasekhar White Dwarf Equation (CWDE). The CWDE is fundamental for understanding the life cycle of a star, and describes the relationship between the density of the white dwarf and its distance from the center. Despite the rise in Scientific Machine Learning frameworks, very less attention has been paid to the systematic applications of the above SciML pillars on astronomy based ODEs. Through robust modeling in the Julia programming language, we show that both Neural ODEs and UDEs can be used effectively for both prediction as well as forecasting of the CWDE. More importantly, we introduce the forecasting breakdown point - the time at which forecasting fails for both Neural ODEs and UDEs. Through a robust hyperparameter optimization testing, we provide insights on the neural network architecture, activation functions and optimizers which provide the best results. This study provides opens a door to investigate the applicability of Scientific Machine Learning frameworks in forecasting tasks for a wide range of scientific domains.

This project explores the application of machine learning techniques for music genre classification using the GTZAN dataset, which contains 100 audio files per genre. Motivated by the growing demand for personalized music recommendations, we focused on classifying five genres-Blues, Classical, Jazz, Hip Hop, and Country-using a variety of algorithms including Logistic Regression, K-Nearest Neighbors (KNN), Random Forest, and Artificial Neural Networks (ANN) implemented via Keras. The ANN model demonstrated the best performance, achieving a validation accuracy of 92.44%. We also analyzed key audio features such as spectral roll-off, spectral centroid, and MFCCs, which helped enhance the model's accuracy. Future work will expand the model to cover all ten genres, investigate advanced methods like Long Short-Term Memory (LSTM) networks and ensemble approaches, and develop a web application for real-time genre classification and playlist generation. This research aims to contribute to improving music recommendation systems and content curation.

Carbon emissions are rising at an alarming rate, posing a significant threat to global efforts to mitigate climate change. Electric vehicles have emerged as a promising solution, but their reliance on lithium-ion batteries introduces the critical challenge of battery degradation. Accurate prediction and forecasting of battery degradation over both short and long time spans are essential for optimizing performance, extending battery life, and ensuring effective long-term energy management. This directly influences the reliability, safety, and sustainability of EVs, supporting their widespread adoption and aligning with key UN SDGs. In this paper, we present a novel approach to the prediction and long-term forecasting of battery degradation using Scientific Machine Learning framework which integrates domain knowledge with neural networks, offering more interpretable and scientifically grounded solutions for both predicting short-term battery health and forecasting degradation over extended periods. This hybrid approach captures both known and unknown degradation dynamics, improving predictive accuracy while reducing data requirements. We incorporate ground-truth data to inform our models, ensuring that both the predictions and forecasts reflect practical conditions. The model achieved MSE of 9.90 with the UDE and 11.55 with the NeuralODE, in experimental data, a loss of 1.6986 with the UDE, and a MSE of 2.49 in the NeuralODE, demonstrating the enhanced precision of our approach. This integration of data-driven insights with SciML's strengths in interpretability and scalability allows for robust battery management. By enhancing battery longevity and minimizing waste, our approach contributes to the sustainability of energy systems and accelerates the global transition toward cleaner, more responsible energy solutions, aligning with the UN's SDG agenda.

Evaluating machine-generated text remains a significant challenge in NLP, especially for non-English languages. Current methodologies, including automated metrics, human assessments, and LLM-based evaluations, predominantly focus on English, revealing a significant gap in multilingual evaluation frameworks. We introduce the Cross Lingual Auto Evaluation (CIA) Suite, an extensible framework that includes evaluator LLMs (Hercule) and a novel test set (Recon) specifically designed for multilingual evaluation. Our test set features 500 human-annotated instructions spanning various task capabilities along with human judgment scores across six languages. This would enable benchmarking of general-purpose multilingual LLMs and facilitate meta-evaluation of Evaluator LLMs. The proposed model, Hercule, is a cross-lingual evaluation model that addresses the scarcity of reference answers in the target language by learning to assign scores to responses based on easily available reference answers in English. Our experiments demonstrate that Hercule aligns more closely with human judgments compared to proprietary models, demonstrating the effectiveness of such cross-lingual evaluation in low resource scenarios. Further, it is also effective in zero-shot evaluation on unseen languages. This study is the first comprehensive examination of cross-lingual evaluation using LLMs, presenting a scalable and effective approach for multilingual assessment. All code, datasets, and models will be publicly available to enable further research in this important area.

Understanding how speech foundation models capture non-verbal cues is crucial for improving their interpretability and adaptability across diverse tasks. In our work, we analyze several prominent models such as Whisper, Seamless, Wav2Vec, HuBERT, and Qwen2-Audio focusing on their learned representations in both paralinguistic and non-paralinguistic tasks from the Dynamic-SUPERB benchmark. Our study addresses three key questions: (1) What non-verbal cues (e.g., speaker intent, emotion, environmental context) are captured? (2) How are these cues represented across different layers of the models? and (3) To what extent can these representations be effectively adapted to downstream tasks? To answer these questions, we first evaluate the models in a zero-shot setting, followed by fine-tuning on layer-wise features extracted from these models. Our results provide insights into the models' capacity for generalization, the characteristics of their layer-wise representations, and the degree of transformation required for downstream task adaptation. Our findings suggest that some of these models perform well on various tasks in zero-shot settings, despite not being explicitly trained for those tasks. We also observe that zero-shot performance correlates with better-learned representations. The analysis of layer-wise features demonstrates that some models exhibit a convex relationship between the separability of the learned representations and model depth, with different layers capturing task-specific features., Comment: 20 Pages

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data., Comment: Preprint

The NOvA collaboration reports cross-section measurements for $\nu_{\mu}$ charged-current interactions with low hadronic energy (maximum kinetic energy of 250 MeV for protons and 175 MeV for pions) in the NOvA Near Detector. The results are presented as a double-differential cross section as a function of the direct observables of the final-state muon kinematics. Results are also presented as a single-differential cross section as a function of the derived square of the four-momentum transfer, $Q^{2}$, and as a function of the derived neutrino energy. The data correspond to an accumulated 8.09$\times10^{20}$ protons-on-target (POT) in the neutrino mode of the NuMI beam, with a narrow band of neutrino energies peaked at 1.8 GeV. The analysis provides a sample of neutrino-nucleus interactions with an enhanced fraction of quasi-elastic and two-particle-two-hole (2p2h) interactions. This enhancement allows quantitative comparisons with various nuclear models. We find strong disagreement between data and theory-based models in various regions of the muon kinematic phase space, especially in the forward muon direction., Comment: 20 pages, 12 figures

Garbage production and littering are persistent global issues that pose significant environmental challenges. Despite large-scale efforts to manage waste through collection and sorting, existing approaches remain inefficient, leading to inadequate recycling and disposal. Therefore, developing advanced AI-based systems is less labor intensive approach for addressing the growing waste problem more effectively. These models can be applied to sorting systems or possibly waste collection robots that may produced in the future. AI models have grown significantly at identifying objects through object detection. This paper reviews the implementation of AI models for classifying trash through object detection, specifically focusing on using YOLO V5 for training and testing. The study demonstrates how YOLO V5 can effectively identify various types of waste, including plastic, paper, glass, metal, cardboard, and biodegradables., Comment: 8 pages, 8 figures

The long-term variability study over a range of black hole (BH) mass systems from the microquasars of stellar-mass black holes to the Active Galactic Nuclei (AGNs) of supermassive black holes, in $\gamma$-rays offers new insights into the physics of relativistic jets. In this work, we investigate the $\gamma$-ray variability of 11 AGNs--including 7 blazars, 2 unclassified blazar candidates (BCUs), 1 radio galaxy (RG), and 1 narrow-line Seyfert 1 galaxy (NLS1) as well as 2 microquasars. We apply a stochastic process known as the Damped Random Walk (DRW) to model the $\sim$15 years of Fermi-LAT light curves. The characteristic timescales observed for AGNs are comparable to those in the accretion disc. Interestingly, the timescales observed in the jet emission of microquasars are similar to those of AGNs, suggesting uniform jet properties across the black hole masses. The observed rest-frame timescales of AGNs overlap with both thermal and non-thermal timescales associated with the jet and accretion disk, respectively, suggesting a scaled relationship between $\tau_{DRW}^{rest}$ and black hole mass ($\rm{M_{BH}}$). While the timescales observed for microquasars deviate significantly from this relationship, nonetheless exhibit a scaled $\tau_{DRW}^{rest}-\rm{M_{BH}}$ relationship using $\gamma$-rays specifically. These findings offer new insights into the origin of jets and the processes driving the emission within them. Additionally, this study hints at a new perspective that the relativistic jets' properties or their production mechanisms may be independent of the black hole mass., Comment: 13 pages, 5 figures, submitted to Journal of High Energy Astrophysics

Consistency regularization is a commonly used practice to encourage the model to generate consistent representation from distorted input features and improve model generalization. It shows significant improvement on various speech applications that are optimized with cross entropy criterion. However, it is not straightforward to apply consistency regularization for the transducer-based approaches, which are widely adopted for speech applications due to the competitive performance and streaming characteristic. The main challenge is from the vast alignment space of the transducer optimization criterion and not all the alignments within the space contribute to the model optimization equally. In this study, we present Transducer Consistency Regularization (TCR), a consistency regularization method for transducer models. We apply distortions such as spec augmentation and dropout to create different data views and minimize the distribution difference. We utilize occupational probabilities to give different weights on transducer output distributions, thus only alignments close to oracle alignments would contribute to the model learning. Our experiments show the proposed method is superior to other consistency regularization implementations and could effectively reduce word error rate (WER) by 4.3\% relatively comparing with a strong baseline on the \textsc{Librispeech} dataset., Comment: 8 pages, 4 figures. Accepted in IEEE Spoken Language Technology Workshop 2024

This work presents the measurements of the cosmic muon charge ratio as a function of full azimuthal angle and momentum within the range of 0.8 to 3.0 GeV/c, using the mini-ICAL detector. The detector, comprising 10 layers of RPCs, has collected cosmic muon data since August 2018 till recent time, at an altitude of 160 m above sea level at the Inter-Institutional Center for High Energy Physics in Madurai, India $(9^\circ56'\,N, 78^\circ00'\,E)$. The muon charge identification is achieved through the use of a magnetic field of strength 1.4 T. The analysis shows that the cosmic muon charge ratio, $R_\mu = N_{\mu^+}/N_{\mu^-}$, ranges from 1.1 to 1.2 and has small dependency on the zenith angle. The charge ratio's dependence on momentum and azimuthal angle is thoroughly examined for a wide range of zenith angle upto $50^\circ$. These measurements are compared with the predictions from various combinations of different hadronic models in CORSIKA extensive air shower simulations., Comment: 19 pages, 15 figures

In climate science, models for global warming and weather prediction face significant challenges due to the limited availability of high-quality data and the difficulty in obtaining it, making data efficiency crucial. In the past few years, Scientific Machine Learning (SciML) models have gained tremendous traction as they can be trained in a data-efficient manner, making them highly suitable for real-world climate applications. Despite this, very little attention has been paid to chaotic climate system modeling utilizing SciML methods. In this paper, we have integrated SciML methods into foundational weather models, where we have enhanced large-scale climate predictions with a physics-informed approach that achieves high accuracy with reduced data. We successfully demonstrate that by combining the interpretability of physical climate models with the computational power of neural networks, SciML models can prove to be a reliable tool for modeling climate. This indicates a shift from the traditional black box-based machine learning modeling of climate systems to physics-informed decision-making, leading to effective climate policy implementation., Comment: 13 pages, 8 figures, 3 tables

Over the last 50 years entangled photon pairs have received attention for use in lowering the flux in two-photon absorption imaging and spectroscopy. Despite this, evidence for entangled two-photon absorption (ETPA) effects remain highly debated, especially at low-fluxes. Here, we structure the transverse spatial correlations of entangled photon pairs to evidence signs of ETPA at room-temperature in organic and inorganic chromophores, in the low-flux regime. We demonstrate our scheme to be robust to common artifacts that have previously hampered detection of ETPA such as linear absorption and background fluorescence, and show that ETPA scales with transverse correlation area and chromophore two-photon cross-sections. Our results present a step towards verifying ETPA and experimentally exploring entangled light-matter interactions., Comment: 21 pages, 9 figures

Double- and single-differential cross sections for inclusive charged-current neutrino-nucleus scattering are reported for the kinematic domain 0 to 2 GeV/c in three-momentum transfer and 0 to 2 GeV in available energy, at a mean muon-neutrino energy of 1.86 GeV. The measurements are based on an estimated 995,760 muon-neutrino CC interactions in the scintillator medium of the NOvA Near Detector. The subdomain populated by 2-particle-2-hole reactions is identified by the cross-section excess relative to predictions for neutrino-nucleus scattering that are constrained by a data control sample. Models for 2-particle-2- hole processes are rated by chi-square comparisons of the predicted-versus-measured muon-neutrino CC inclusive cross section over the full phase space and in the restricted subdomain. Shortfalls are observed in neutrino generator predictions obtained using the theory-based Val`encia and SuSAv2 2p2h models., Comment: 20 pages, 14 figures

Speaker verification systems have seen significant advancements with the introduction of Multi-scale Feature Aggregation (MFA) architectures, such as MFA-Conformer and ECAPA-TDNN. These models leverage information from various network depths by concatenating intermediate feature maps before the pooling and projection layers, demonstrating that even shallower feature maps encode valuable speaker-specific information. Building upon this foundation, we propose a Multi-scale Feature Contrastive (MFCon) loss that directly enhances the quality of these intermediate representations. Our MFCon loss applies contrastive learning to all feature maps within the network, encouraging the model to learn more discriminative representations at the intermediate stage itself. By enforcing better feature map learning, we show that the resulting speaker embeddings exhibit increased discriminative power. Our method achieves a 9.05% improvement in equal error rate (EER) compared to the standard MFA-Conformer on the VoxCeleb-1O test set.

Neural multi-channel speech enhancement models, in particular those based on the U-Net architecture, demonstrate promising performance and generalization potential. These models typically encode input channels independently, and integrate the channels during later stages of the network. In this paper, we propose a novel modification of these models by incorporating relative information from the outset, where each channel is processed in conjunction with a reference channel through stacking. This input strategy exploits comparative differences to adaptively fuse information between channels, thereby capturing crucial spatial information and enhancing the overall performance. The experiments conducted on the CHiME-3 dataset demonstrate improvements in speech enhancement metrics across various architectures.

Conventional federated learning frameworks suffer from several challenges including performance bottlenecks at the central aggregation server, data bias, poor model convergence, and exposure to model poisoning attacks, and limited trust in the centralized infrastructure. In the current paper, a novel game theory-based approach called pFedGame is proposed for decentralized federated learning, best suitable for temporally dynamic networks. The proposed algorithm works without any centralized server for aggregation and incorporates the problem of vanishing gradients and poor convergence over temporally dynamic topology among federated learning participants. The solution comprises two sequential steps in every federated learning round, for every participant. First, it selects suitable peers for collaboration in federated learning. Secondly, it executes a two-player constant sum cooperative game to reach convergence by applying an optimal federated learning aggregation strategy. Experiments performed to assess the performance of pFedGame in comparison to existing methods in decentralized federated learning have shown promising results with accuracy higher than 70% for heterogeneous data.

We introduce a novel, general-purpose audio generation framework specifically designed for anomaly detection and localization. Unlike existing datasets that predominantly focus on industrial and machine-related sounds, our framework focuses a broader range of environments, particularly useful in real-world scenarios where only audio data are available, such as in video-derived or telephonic audio. To generate such data, we propose a new method inspired by the LLM-Modulo framework, which leverages large language models(LLMs) as world models to simulate such real-world scenarios. This tool is modular allowing a plug-and-play approach. It operates by first using LLMs to predict plausible real-world scenarios. An LLM further extracts the constituent sounds, the order and the way in which these should be merged to create coherent wholes. Much like the LLM-Modulo framework, we include rigorous verification of each output stage, ensuring the reliability of the generated data. The data produced using the framework serves as a benchmark for anomaly detection applications, potentially enhancing the performance of models trained on audio data, particularly in handling out-of-distribution cases. Our contributions thus fill a critical void in audio anomaly detection resources and provide a scalable tool for generating diverse, realistic audio data., Comment: 9 pages, under review

We explore the chaotic dynamics of a large one-dimensional lattice of coupled maps with diffusive coupling of varying strength using the covariant Lyapunov vectors (CLVs). Using a lattice of diffusively coupled quadratic maps we quantify the growth of spatial structures in the chaotic dynamics as the strength of diffusion is increased. When the diffusion strength is increased from zero, we find that the leading Lyapunov exponent decreases rapidly from a positive value to zero to yield a small window of periodic dynamics which is then followed by chaotic dynamics. For values of the diffusion strength beyond the window of periodic dynamics, the leading Lyapunov exponent does not vary significantly with the strength of diffusion with the exception of a small variation for the largest diffusion strengths we explore. The Lyapunov spectrum and fractal dimension are described analytically as a function of the diffusion strength using the eigenvalues of the coupling operator. The spatial features of the CLVs are quantified and compared with the eigenvectors of the coupling operator. The chaotic dynamics are composed entirely of physical modes for all of the conditions we explore. The leading CLV is highly localized and the localization decreases with increasing strength of the spatial coupling. The violation of the dominance of Oseledets splitting indicates that the entanglement of pairs of CLVs becomes more significant between neighboring CLVs as the strength of the diffusion is increased., Comment: 33 pages, 15 figures

Image tokenizers are crucial for visual generative models, e.g., diffusion models (DMs) and autoregressive (AR) models, as they construct the latent representation for modeling. Increasing token length is a common approach to improve the image reconstruction quality. However, tokenizers with longer token lengths are not guaranteed to achieve better generation quality. There exists a trade-off between reconstruction and generation quality regarding token length. In this paper, we investigate the impact of token length on both image reconstruction and generation and provide a flexible solution to the tradeoff. We propose ImageFolder, a semantic tokenizer that provides spatially aligned image tokens that can be folded during autoregressive modeling to improve both generation efficiency and quality. To enhance the representative capability without increasing token length, we leverage dual-branch product quantization to capture different contexts of images. Specifically, semantic regularization is introduced in one branch to encourage compacted semantic information while another branch is designed to capture the remaining pixel-level details. Extensive experiments demonstrate the superior quality of image generation and shorter token length with ImageFolder tokenizer., Comment: Code: https://github.com/lxa9867/ImageFolder

Encoding quantum information within bosonic modes offers a promising direction for hardware-efficient and fault-tolerant quantum information processing. However, achieving high-fidelity universal control over the bosonic degree of freedom using native photonic hardware remains a challenge. Here, we propose an architecture to prepare and perform logical quantum operations on arbitrary multimode multi-photon states using a quantum photonic neural network. Central to our approach is the optical nonlinearity, which is realized through strong light-matter interaction with a three-level Lambda atomic system. The dynamics of this interaction are confined to the single-mode subspace, enabling the construction of high-fidelity quantum gates. This nonlinearity functions as a photon-number selective phase gate, which facilitates the construction of a universal gate set and serves as the element-wise activation function in our neural network architecture. Through numerical simulations, we demonstrate the versatility of our approach by executing tasks that are key to logical quantum information processing. The network is able to deterministically prepare a wide array of multimode multi-photon states, including essential resource states. We also show that the architecture is capable of encoding and performing logical operations on bosonic error-correcting codes. Additionally, by adapting components of our architecture, error-correcting circuits can be built to protect bosonic codes. The proposed architecture paves the way for near-term quantum photonic processors that enable error-corrected quantum computation, and can be achieved using present-day integrated photonic hardware., Comment: 21 pages including supplement, 12 figures total

Two matrices are said to be principal minor equivalent if they have equal corresponding principal minors of all orders. We give a characterization of principal minor equivalence and a deterministic polynomial time algorithm to check if two given matrices are principal minor equivalent. Earlier such results were known for certain special cases like symmetric matrices, skew-symmetric matrices with {0, 1, -1}-entries, and matrices with no cuts (i.e., for any non-trivial partition of the indices, the top right block or the bottom left block must have rank more than 1). As an immediate application, we get an algorithm to check if the determinantal point processes corresponding to two given kernel matrices (not necessarily symmetric) are the same. As another application, we give a deterministic polynomial-time test to check equality of two multivariate polynomials, each computed by a symbolic determinant with a rank 1 constraint on coefficient matrices.

Super-Earths are highly irradiated, small planets with bulk densities approximately consistent with Earth. We construct combined interior-atmosphere models of super-Earths that trace the partitioning of water throughout a planet, including an iron-rich core, silicate-rich mantle, and steam atmosphere. We compare these models with exoplanet observations to infer a $1\sigma$ upper limit on total water mass fraction of $\lesssim 3\%$ at the population level. We consider end-member scenarios that may change this value, including the efficiency of mantle outgassing and escape of high mean-molecular weight atmospheres. Although our constraints are agnostic as to the origin of water, we show that our upper limits are consistent with its production via chemical reactions of primordial hydrogen-dominated atmospheres with magma oceans. This mechanism has also been hypothesised to explain Earth's water content, possibly pointing to a unified channel for the origins of water on small terrestrial planets., Comment: Submitted to ApJ

Automatic Speech Recognition (ASR) systems must be robust to the myriad types of noises present in real-world environments including environmental noise, room impulse response, special effects as well as attacks by malicious actors (adversarial attacks). Recent works seek to improve accuracy and robustness by developing novel Deep Neural Networks (DNNs) and curating diverse training datasets for them, while using relatively simple acoustic features. While this approach improves robustness to the types of noise present in the training data, it confers limited robustness against unseen noises and negligible robustness to adversarial attacks. In this paper, we revisit the approach of earlier works that developed acoustic features inspired by biological auditory perception that could be used to perform accurate and robust ASR. In contrast, Specifically, we evaluate the ASR accuracy and robustness of several biologically inspired acoustic features. In addition to several features from prior works, such as gammatone filterbank features (GammSpec), we also propose two new acoustic features called frequency masked spectrogram (FreqMask) and difference of gammatones spectrogram (DoGSpec) to simulate the neuro-psychological phenomena of frequency masking and lateral suppression. Experiments on diverse models and datasets show that (1) DoGSpec achieves significantly better robustness than the highly popular log mel spectrogram (LogMelSpec) with minimal accuracy degradation, and (2) GammSpec achieves better accuracy and robustness to non-adversarial noises from the Speech Robust Bench benchmark, but it is outperformed by DoGSpec against adversarial attacks., Comment: submitted to ICASSP 2025

Neural codecs have become crucial to recent speech and audio generation research. In addition to signal compression capabilities, discrete codecs have also been found to enhance downstream training efficiency and compatibility with autoregressive language models. However, as extensive downstream applications are investigated, challenges have arisen in ensuring fair comparisons across diverse applications. To address these issues, we present a new open-source platform ESPnet-Codec, which is built on ESPnet and focuses on neural codec training and evaluation. ESPnet-Codec offers various recipes in audio, music, and speech for training and evaluation using several widely adopted codec models. Together with ESPnet-Codec, we present VERSA, a standalone evaluation toolkit, which provides a comprehensive evaluation of codec performance over 20 audio evaluation metrics. Notably, we demonstrate that ESPnet-Codec can be integrated into six ESPnet tasks, supporting diverse applications., Comment: Accepted by SLT

Accurately detecting and tracking high-speed, small objects, such as balls in sports videos, is challenging due to factors like motion blur and occlusion. Although recent deep learning frameworks like TrackNetV1, V2, and V3 have advanced tennis ball and shuttlecock tracking, they often struggle in scenarios with partial occlusion or low visibility. This is primarily because these models rely heavily on visual features without explicitly incorporating motion information, which is crucial for precise tracking and trajectory prediction. In this paper, we introduce an enhancement to the TrackNet family by fusing high-level visual features with learnable motion attention maps through a motion-aware fusion mechanism, effectively emphasizing the moving ball's location and improving tracking performance. Our approach leverages frame differencing maps, modulated by a motion prompt layer, to highlight key motion regions over time. Experimental results on the tennis ball and shuttlecock datasets show that our method enhances the tracking performance of both TrackNetV2 and V3. We refer to our lightweight, plug-and-play solution, built on top of the existing TrackNet, as TrackNetV4., Comment: Research report

Understanding satire and humor is a challenging task for even current Vision-Language models. In this paper, we propose the challenging tasks of Satirical Image Detection (detecting whether an image is satirical), Understanding (generating the reason behind the image being satirical), and Completion (given one half of the image, selecting the other half from 2 given options, such that the complete image is satirical) and release a high-quality dataset YesBut, consisting of 2547 images, 1084 satirical and 1463 non-satirical, containing different artistic styles, to evaluate those tasks. Each satirical image in the dataset depicts a normal scenario, along with a conflicting scenario which is funny or ironic. Despite the success of current Vision-Language Models on multimodal tasks such as Visual QA and Image Captioning, our benchmarking experiments show that such models perform poorly on the proposed tasks on the YesBut Dataset in Zero-Shot Settings w.r.t both automated as well as human evaluation. Additionally, we release a dataset of 119 real, satirical photographs for further research. The dataset and code are available at https://github.com/abhi1nandy2/yesbut_dataset., Comment: EMNLP 2024 Main (Long), 18 pages, 14 figures, 12 tables

Capturing audio signals with specific directivity patterns is essential in speech communication. This study presents a deep neural network (DNN)-based approach to directional filtering, alleviating the need for explicit signal models. More specifically, our proposed method uses a DNN to estimate a single-channel complex mask from the signals of a microphone array. This mask is then applied to a reference microphone to render a signal that exhibits a desired directivity pattern. We investigate the training dataset composition and its effect on the directivity realized by the DNN during inference. Using a relatively small DNN, the proposed method is found to approximate the desired directivity pattern closely. Additionally, it allows for the realization of higher-order directivity patterns using a small number of microphones, which is a difficult task for linear and parametric directional filtering., Comment: Presented at the International Workshop on Acoustic Signal Enhancement (IWAENC), 2024

The remarkable advancements in artificial intelligence (AI), primarily driven by deep neural networks, are facing challenges surrounding unsustainable computational trajectories, limited robustness, and a lack of explainability. To develop next-generation cognitive AI systems, neuro-symbolic AI emerges as a promising paradigm, fusing neural and symbolic approaches to enhance interpretability, robustness, and trustworthiness, while facilitating learning from much less data. Recent neuro-symbolic systems have demonstrated great potential in collaborative human-AI scenarios with reasoning and cognitive capabilities. In this paper, we aim to understand the workload characteristics and potential architectures for neuro-symbolic AI. We first systematically categorize neuro-symbolic AI algorithms, and then experimentally evaluate and analyze them in terms of runtime, memory, computational operators, sparsity, and system characteristics on CPUs, GPUs, and edge SoCs. Our studies reveal that neuro-symbolic models suffer from inefficiencies on off-the-shelf hardware, due to the memory-bound nature of vector-symbolic and logical operations, complex flow control, data dependencies, sparsity variations, and limited scalability. Based on profiling insights, we suggest cross-layer optimization solutions and present a hardware acceleration case study for vector-symbolic architecture to improve the performance, efficiency, and scalability of neuro-symbolic computing. Finally, we discuss the challenges and potential future directions of neuro-symbolic AI from both system and architectural perspectives., Comment: 14 pages, 11 figures, 7 tables; IEEE Transactions on Circuits and Systems for Artificial Intelligence (TCASAI), 2024

While there has been progress towards aligning Large Language Models (LLMs) with human values and ensuring safe behaviour at inference time, safety-aligned LLMs are known to be vulnerable to training-time attacks such as supervised fine-tuning (SFT) on harmful datasets. In this paper, we ask if LLMs are vulnerable to adversarial reinforcement learning. Motivated by this goal, we propose Reverse Preference Attacks (RPA), a class of attacks to make LLMs learn harmful behavior using adversarial reward during reinforcement learning from human feedback (RLHF). RPAs expose a critical safety gap of safety-aligned LLMs in RL settings: they easily explore the harmful text generation policies to optimize adversarial reward. To protect against RPAs, we explore a host of mitigation strategies. Leveraging Constrained Markov-Decision Processes, we adapt a number of mechanisms to defend against harmful fine-tuning attacks into the RL setting. Our experiments show that ``online" defenses that are based on the idea of minimizing the negative log likelihood of refusals -- with the defender having control of the loss function -- can effectively protect LLMs against RPAs. However, trying to defend model weights using ``offline" defenses that operate under the assumption that the defender has no control over the loss function are less effective in the face of RPAs. These findings show that attacks done using RL can be used to successfully undo safety alignment in open-weight LLMs and use them for malicious purposes.

Large language models (LLMs) are increasingly used in daily work. In this paper, we analyze whether training in prompt engineering can improve the interactions of users with LLMs. For this, we conducted a field experiment where we asked journalists to write short texts before and after training in prompt engineering. We then analyzed the effect of training on three dimensions: (1) the user experience of journalists when interacting with LLMs, (2) the accuracy of the texts (assessed by a domain expert), and (3) the reader perception, such as clarity, engagement, and other text quality dimensions (assessed by non-expert readers). Our results show: (1) Our training improved the perceived expertise of journalists but also decreased the perceived helpfulness of LLM use. (2) The effect on accuracy varied by the difficulty of the task. (3) There is a mixed impact of training on reader perception across different text quality dimensions.

Consensus control of multiagent systems arises in various robotic applications such as rendezvous and formation control. For example, to compute the control inputs of individual agents, the difference in the positions in aligned coordinate frames i.e., the pairwise displacements are typically measured. However, the local coordinate frames might be subject to rotational ambiguities, such as a rotation or a reflection, particularly if the positions of the agent are not directly observed but reconstructed from e.g. pairwise Euclidean distances. This rotational ambiguity causes stability issues in practice, as agents have rotated perceptions of the environment. In this work, we conduct a thorough analysis of the stability in the presence of rotational ambiguities in several scenarios including e.g., proper and improper rotation, and the homogeneity of rotations. We give stability criteria and stability margin on the rotations, which are numerically verified with two traditional examples of consensus control., Comment: This work has been submitted to the IEEE for possible publication

The growing popularity of multi-channel wearable devices, such as smart glasses, has led to a surge of applications such as targeted speech recognition and enhanced hearing. However, current approaches to solve these tasks use independently trained models, which may not benefit from large amounts of unlabeled data. In this paper, we propose M-BEST-RQ, the first multi-channel speech foundation model for smart glasses, which is designed to leverage large-scale self-supervised learning (SSL) in an array-geometry agnostic approach. While prior work on multi-channel speech SSL only evaluated on simulated settings, we curate a suite of real downstream tasks to evaluate our model, namely (i) conversational automatic speech recognition (ASR), (ii) spherical active source localization, and (iii) glasses wearer voice activity detection, which are sourced from the MMCSG and EasyCom datasets. We show that a general-purpose M-BEST-RQ encoder is able to match or surpass supervised models across all tasks. For the conversational ASR task in particular, using only 8 hours of labeled speech, our model outperforms a supervised ASR baseline that is trained on 2000 hours of labeled data, which demonstrates the effectiveness of our approach., Comment: In submission to IEEE ICASSP 2025

Organic spintronics has drawn the interest of the science community due to various applications in spin-valve devices. However, an efficient room-temperature Organic Spin Valve device has not been experimentally realized due to the complicated spin transport at the metal-organic interfaces. The present study focuses on a comprehensive understanding of the interfacial properties essential for advancing device performance and functionality. The structural and magnetic properties of the ultra-thin Cobalt (Co) films deposited on the fullerene (C60) layer are studied to investigate the origin of magnetic anisotropy in the metal-organic bilayer structures. Due to the mechanical softness of C60, penetration of ferromagnetic Co atoms inside the C60 film is confirmed by the X-ray reflectivity and Secondary Ion Mass Spectroscopy measurements. Grazing incidence small-angle X-ray scattering and atomic force microscopy provided information regarding the structural and morphological properties of the Co/C60 bilayers, angular dependent Magneto-optic Kerr effect measurements with varying Co layer thickness provided information about the growth-induced uniaxial magnetic anisotropy. In contrast to the inorganic silicon substrates, magnetic anisotropy in Co film tends to develop at 25 {\AA} thickness on the C60 layer, which further increases with the thickness of Cobalt. The anomalous behavior in coercivity and remanence variation along the nominal hard axis is explained by a two-grain Stoner-Wohlfarth model with intergranular exchange coupling. It is further confirmed by a non-uniform spatial distribution of magnetic domains investigated through Kerr microscopy. These anomalies could be attributed to the distribution of magneto-crystalline anisotropy and inhomogeneous strain caused by the formation of a diffused layer at the Co/C60 interface., Comment: 18 pages, 13 Figures