Search

Digital Twin technology creates virtual replicas of physical objects, processes, or systems by replicating their properties, data, and behaviors. This advanced technology offers a range of intelligent functionalities, such as modeling, simulation, and data-driven decision-making, that facilitate design optimization, performance estimation, and monitoring operations. Forecasting plays a pivotal role in Digital Twin technology, as it enables the prediction of future outcomes, supports informed decision-making, minimizes risks, driving improvements in efficiency, productivity, and cost reduction. Recently, Digital Twin technology has leveraged Graph forecasting techniques in large-scale complex sensor networks to enable accurate forecasting and simulation of diverse scenarios, fostering proactive and data-driven decision making. However, existing Graph forecasting techniques lack scalability for many real-world applications. They have limited ability to adapt to non-stationary environments, retain past knowledge, lack a mechanism to capture the higher order spatio-temporal dynamics, and estimate uncertainty in model predictions. To surmount the challenges, we introduce a hybrid architecture that enhances the hypergraph representation learning backbone by incorporating fast adaptation to new patterns and memory-based retrieval of past knowledge. This balance aims to improve the slowly-learned backbone and achieve better performance in adapting to recent changes. In addition, it models the time-varying uncertainty of multi-horizon forecasts, providing estimates of prediction uncertainty. Our forecasting architecture has been validated through ablation studies and has demonstrated promising results across multiple benchmark datasets, surpassing state-ofthe-art forecasting methods by a significant margin., Comment: Paper accepted at AI for Digital Twins and Cyber-Physical Applications Workshop, International Joint Conferences on Artificial Intelligence(IJCAI-23). arXiv admin note: text overlap with arXiv:2408.12409

Anomaly detection is fundamental yet, challenging problem with practical applications in industry. The current approaches neglect the higher-order dependencies within the networks of interconnected sensors in the high-dimensional time series(multisensor data) for anomaly detection. To this end, we present a self-adapting anomaly detection framework for joint learning of (a) discrete hypergraph structure and (b) modeling the temporal trends and spatial relations among the interdependent sensors using the hierarchical encoder-decoder architecture to overcome the challenges. The hypergraph representation learning-based framework exploits the relational inductive biases in the hypergraph-structured data to learn the pointwise single-step-ahead forecasts through the self-supervised autoregressive task and predicts the anomalies based on the forecast error. Furthermore, our framework incentivizes learning the anomaly-diagnosis ontology through a differentiable approach. It derives the anomaly information propagation-based computational hypergraphs for root cause analysis and provides recommendations through an offline, optimal predictive control policy to remedy an anomaly. We conduct extensive experiments to evaluate the proposed method on the benchmark datasets for fair and rigorous comparison with the popular baselines. The proposed method outperforms the baseline models and achieves SOTA performance. We report the ablation studies to support the efficacy of the framework., Comment: 16 pages, 10 figure, Accepted at IEEE International Conference on Big Data 2022, Osaka, Japan

Material characterization using electron micrographs is a crucial but challenging task with applications in various fields, such as semiconductors, quantum materials, batteries, etc. The challenges in categorizing electron micrographs include but are not limited to the complexity of patterns, high level of detail, and imbalanced data distribution(long-tail distribution). Existing methods have difficulty in modeling the complex relational structure in electron micrographs, hindering their ability to effectively capture the complex relationships between different spatial regions of micrographs. We propose a hypergraph neural network(HgNN) backbone architecture, a conceptually alternative approach, to better model the complex relationships in electron micrographs and improve material characterization accuracy. By utilizing cost-effective GPU hardware, our proposed framework outperforms popular baselines. The results of the ablation studies demonstrate that the proposed framework is effective in achieving state-of-the-art performance on benchmark datasets and efficient in terms of computational and memory requirements for handling large-scale electron micrograph-based datasets., Comment: 21 pages, Accepted in PML4DC Workshop at International Conference on Learning Representations (ICLR) 2023

Sex chromosomes have independently evolved in species with separate sexes in most lineages across the tree of life. However, the well-accepted canonical model of sex chromosome evolution is not universally supported. There is no single trajectory for sex chromosome formation and evolution across the tree of life, suggesting the underlying mechanisms and evolutionary forces are diverse and lineage specific. We review the diversity of sex chromosome systems, describe the canonical model of sex chromosome evolution, and summarize studies challenging various aspects of this model. They include evidence that many lineages experience frequent sex chromosome turnovers or maintain homomorphic sex chromosomes over long periods of time, suggesting sex chromosome degeneration is not inevitable. Sometimes the sex-limited Y/W chromosomes expand before they contract in size. Both transposable elements and gene gains could contribute to this size expansion, which further challenges gene loss being the hallmark of sex chromosome degeneration. Finally, empirical support for the role of sexually antagonistic selection as a driver of recombination suppression on sex chromosomes remains elusive. We summarize models that result in loss of recombination without invoking sexually antagonistic selection, which have not been empirically verified yet, and suggest future avenues for sex chromosome research., Comment: This has been accepted as a Book chapter for Encyclopedia of Evolutionary Biology book 2nd Edition

Characterization of materials via electron micrographs is an important and challenging task in several materials processing industries. Classification of electron micrographs is complex due to the high intra-class dissimilarity, high inter-class similarity, and multi-spatial scales of patterns. However, existing methods are ineffective in learning complex image patterns. We propose an effective end-to-end electron micrograph representation learning-based framework for nanomaterial identification to overcome the challenges. We demonstrate that our framework outperforms the popular baselines on the open-source datasets in nanomaterials-based identification tasks. The ablation studies are reported in great detail to support the efficacy of our approach., Comment: 12 pages, 10 figures, Accepted in a ACM SIGKDD 2022 Workshop on Machine Learning for Materials

Distilling conversational skills into Small Language Models (SLMs) with approximately 1 billion parameters presents significant challenges. Firstly, SLMs have limited capacity in their model parameters to learn extensive knowledge compared to larger models. Secondly, high-quality conversational datasets are often scarce, small, and domain-specific. Addressing these challenges, we introduce a novel data distillation framework named CoDi (short for Conversational Distillation, pronounced "Cody"), allowing us to synthesize large-scale, assistant-style datasets in a steerable and diverse manner. Specifically, while our framework is task agnostic at its core, we explore and evaluate the potential of CoDi on the task of conversational grounded reasoning for question answering. This is a typical on-device scenario for specialist SLMs, allowing for open-domain model responses, without requiring the model to "memorize" world knowledge in its limited weights. Our evaluations show that SLMs trained with CoDi-synthesized data achieve performance comparable to models trained on human-annotated data in standard metrics. Additionally, when using our framework to generate larger datasets from web data, our models surpass larger, instruction-tuned models in zero-shot conversational grounded reasoning tasks., Comment: 13 pages

Large Language Models (LLMs) have revolutionised the capability of AI models in comprehending and generating natural language text. They are increasingly being used to empower and deploy agents in real-world scenarios, which make decisions and take actions based on their understanding of the context. Therefore researchers, policy makers and enterprises alike are working towards ensuring that the decisions made by these agents align with human values and user expectations. That being said, human values and decisions are not always straightforward to measure and are subject to different cognitive biases. There is a vast section of literature in Behavioural Science which studies biases in human judgements. In this work we report an ongoing investigation on alignment of LLMs with human judgements affected by order bias. Specifically, we focus on a famous human study which showed evidence of order effects in similarity judgements, and replicate it with various popular LLMs. We report the different settings where LLMs exhibit human-like order effect bias and discuss the implications of these findings to inform the design and development of LLM based applications., Comment: Accepted at The First Workshop on AI Behavioral Science (AIBS 2024), held in conjunction with KDD 2024

Time series modeling is crucial for many applications, however, it faces challenges such as complex spatio-temporal dependencies and distribution shifts in learning from historical context to predict task-specific outcomes. To address these challenges, we propose a novel approach using an agentic Retrieval-Augmented Generation (RAG) framework for time series analysis. The framework leverages a hierarchical, multi-agent architecture where the master agent orchestrates specialized sub-agents and delegates the end-user request to the relevant sub-agent. The sub-agents utilize smaller, pre-trained language models (SLMs) customized for specific time series tasks through fine-tuning using instruction tuning and direct preference optimization, and retrieve relevant prompts from a shared repository of prompt pools containing distilled knowledge about historical patterns and trends to improve predictions on new data. Our proposed modular, multi-agent RAG approach offers flexibility and achieves state-of-the-art performance across major time series tasks by tackling complex challenges more effectively than task-specific customized methods across benchmark datasets., Comment: Paper was accepted for Undergraduate Consortium at ACM KDD, 2024. Please find the link: https://kdd2024.kdd.org/undergraduate-consortium/

Pre-trained large language models (PLLMs) like OpenAI ChatGPT and Google Gemini face challenges such as inaccurate factual recall, hallucinations, biases, and future data leakage for temporal Knowledge Graph (tKG) forecasting. To address these issues, we introduce sLA-tKGF (small-scale language assistant for tKG forecasting), which utilizes Retrieval-Augmented Generation (RAG) aided, custom-trained small-scale language models through a tabula rasa approach from scratch for effective tKG forecasting. Our framework constructs knowledge-infused prompts with relevant historical data from tKGs, web search results, and PLLMs-generated textual descriptions to understand historical entity relationships prior to the target time. It leverages these external knowledge-infused prompts for deeper understanding and reasoning of context-specific semantic and temporal information to zero-shot prompt small-scale language models for more accurate predictions of future events within tKGs. It reduces hallucinations and mitigates distributional shift challenges through comprehending changing trends over time. As a result, it enables more accurate and contextually grounded forecasts of future events while minimizing computational demands. Rigorous empirical studies demonstrate our framework robustness, scalability, and state-of-the-art (SOTA) performance on benchmark datasets with interpretable and trustworthy tKG forecasting., Comment: Paper was accepted at ACM KDD -2024 -- Undergraduate Consortium. Please find the link: https://kdd2024.kdd.org/undergraduate-consortium/

Molecule design is a multifaceted approach that leverages computational methods and experiments to optimize molecular properties, fast-tracking new drug discoveries, innovative material development, and more efficient chemical processes. Recently, text-based molecule design has emerged, inspired by next-generation AI tasks analogous to foundational vision-language models. Our study explores the use of knowledge-augmented prompting of large language models (LLMs) for the zero-shot text-conditional de novo molecular generation task. Our approach uses task-specific instructions and a few demonstrations to address distributional shift challenges when constructing augmented prompts for querying LLMs to generate molecules consistent with technical descriptions. Our framework proves effective, outperforming state-of-the-art (SOTA) baseline models on benchmark datasets., Comment: Paper was accepted at R0-FoMo: Robustness of Few-shot and Zero-shot Learning in Foundation Models, NeurIPS-2023. Please find the links: https://sites.google.com/view/r0-fomo/accepted-papers?authuser=0 and https://neurips.cc/virtual/2023/workshop/66517

Synergy between evolutionary dynamics of cooperation and fluctuating state of shared resource being consumed by the cooperators is essential for averting the tragedy of the commons. Not only in humans, but also in the cognitively-limited organisms, this interplay between the resource and the cooperation is ubiquitously witnessed. The strategically interacting players engaged in such game-environment feedback scenarios naturally pick strategies based on their perception of the environmental state. Such perception invariably happens through some sensory information channels that the players are endowed with. The unfortunate reality is that any sensory channel must be noisy due to various factors; consequently, the perception of the environmental state becomes faulty rendering the players incapable of adopting the strategy that they otherwise would. Intriguingly, situation is not as bad as it sounds. Here we introduce the hitherto neglected information channel between players and the environment into the paradigm of stochastic evolutionary games with a view to bringing forward the counterintuitive possibility of emergence and sustenance of cooperation on account of the noise in the channel. Our primary study is in the simplest non-trivial setting of two-state stochastically fluctuating resource harnessed by a large unstructured population of cooperators and defectors adopting either memory-1 strategies or reactive strategies while engaged in repeated two-player interactions. The effect of noisy information channel in enhancing the cooperation in reactive-strategied population is unprecedented. We find that the propensity of cooperation in the population is inversely related to the mutual information (normalized by the channel capacity) of the corresponding information channel.

Battery life estimation is critical for optimizing battery performance and guaranteeing minimal degradation for better efficiency and reliability of battery-powered systems. The existing methods to predict the Remaining Useful Life(RUL) of Lithium-ion Batteries (LiBs) neglect the relational dependencies of the battery parameters to model the nonlinear degradation trajectories. We present the Battery GraphNets framework that jointly learns to incorporate a discrete dependency graph structure between battery parameters to capture the complex interactions and the graph-learning algorithm to model the intrinsic battery degradation for RUL prognosis. The proposed method outperforms several popular methods by a significant margin on publicly available battery datasets and achieves SOTA performance. We report the ablation studies to support the efficacy of our approach., Comment: Accepted in Workshop on Graph Learning for Industrial Applications : Finance, Crime Detection, Medicine, and Social Media (NeurIPS 2022)

Generalized Category Discovery (GCD) is a challenging task in which, given a partially labelled dataset, models must categorize all unlabelled instances, regardless of whether they come from labelled categories or from new ones. In this paper, we challenge a remaining assumption in this task: that all images share the same domain. Specifically, we introduce a new task and method to handle GCD when the unlabelled data also contains images from different domains to the labelled set. Our proposed `HiLo' networks extract High-level semantic and Low-level domain features, before minimizing the mutual information between the representations. Our intuition is that the clusterings based on domain information and semantic information should be independent. We further extend our method with a specialized domain augmentation tailored for the GCD task, as well as a curriculum learning approach. Finally, we construct a benchmark from corrupted fine-grained datasets as well as a large-scale evaluation on DomainNet with real-world domain shifts, reimplementing a number of GCD baselines in this setting. We demonstrate that HiLo outperforms SoTA category discovery models by a large margin on all evaluations., Comment: 39 pages, 9 figures, 26 tables

This paper presents a unified algorithm for motion and force control for a six degree-of-freedom spatial manipulator. The motion-force controller performs trajectory tracking, maneuvering the manipulator's end-effector through desired position, orientations and rates. When contacting an obstacle or target object, the force module of the controller restricts the manipulator movements with a novel force exertion method, which prevents damage to the manipulator, the end-effector, and the objects during the contact or collision. The core strategy presented in this paper is to design the linear acceleration for the end-effector which ensures both trajectory tracking and restriction of any contact force at the end-effector. The design of the controller is validated through numerical simulations and digital twin validation.

We study the entanglement structure of the Greenberger-Horne-Zeilinger (GHZ) state as it thermalizes under a strongly-symmetric quantum channel describing the Metropolis-Hastings dynamics for the $d$-dimensional classical Ising model at inverse temperature $\beta$. This channel outputs the classical Gibbs state when acting on a product state in the computational basis. When applying this channel to a GHZ state in spatial dimension $d>1$, the resulting mixed state changes character at the Ising phase transition temperature from being long-range entangled to short-range-entangled as temperature increases. Nevertheless, we show that the topological entanglement negativity of a large region is insensitive to this transition and takes the same value as that of the pure GHZ state at any finite temperature $\beta>0$. We establish this result by devising a local operations and classical communication (LOCC) ``decoder" that provides matching lower and upper bounds on the negativity in the thermodynamic limit which may be of independent interest. This perspective connects the negativity to an error-correction problem on the $(d-1)$-dimensional bipartitioning surface and explains the persistent negativity in certain correlated noise models found in previous studies. Numerical results confirm our analysis., Comment: 7+5 pages, 5 figures

An increasing number of classification approaches have been developed to address the issue of image rebroadcast and recapturing, a standard attack strategy in insurance frauds, face spoofing, and video piracy. However, most of them neglected scale variations and domain generalization scenarios, performing poorly in instances involving domain shifts, typically made worse by inter-domain and cross-domain scale variances. To overcome these issues, we propose a cascaded data augmentation and SWIN transformer domain generalization framework (DAST-DG) in the current research work Initially, we examine the disparity in dataset representation. A feature generator is trained to make authentic images from various domains indistinguishable. This process is then applied to recaptured images, creating a dual adversarial learning setup. Extensive experiments demonstrate that our approach is practical and surpasses state-of-the-art methods across different databases. Our model achieves an accuracy of approximately 82\% with a precision of 95\% on high-variance datasets., Comment: 11 pages, 10 figures, 9 tables

Achieving high quantum efficiency (QE) with low dark count is essential for highly sensitive photodetectors (PDs), including single photon avalanche detectors (SPADs). However, high QE requires a thicker absorber region, which leads to high dark current and noise, which in turn affects the detectivity of PDs and the photodetection efficiency and dark count of SPADs.The holy grail of photodetector and avalanche photodiode designs is to achieve highest QE with thinnest absorber and still enable large avalanche to gain as needed. We have developed a new design paradigm which exploits the coupling between dielectric Mie resonance and transverse propagating waves in thin layers. The Mie resonance launches the incident light at an angle in an ultrathin absorber, and when coupled to transverse waves, the light propagates laterally and is fully absorbed owing to the longer optical path. Consequently, with appropriate choice of materials for a chosen wavelength, a high absorption(~90%) within typically <100 nm absorber thickness is possible. For illustration, we apply our approach to design Si-based detector operating at 810 nm and InGaAs-based detector operating at 1550 nm and predict that the dark current at room temperature is reduced at least by two orders of magnitude. In addition, the lateral distances are often in a few microns and hence these designs can potentially enable avalanching for a large optical gain., Comment: 6 figures

In the rapidly evolving realm of artificial intelligence, deploying large language models (LLMs) poses increasingly pressing computational and environmental challenges. This paper introduces MELODI - Monitoring Energy Levels and Optimization for Data-driven Inference - a multifaceted framework crafted to monitor and analyze the energy consumed during LLM inference processes. MELODI enables detailed observations of power consumption dynamics and facilitates the creation of a comprehensive dataset reflective of energy efficiency across varied deployment scenarios. The dataset, generated using MELODI, encompasses a broad spectrum of LLM deployment frameworks, multiple language models, and extensive prompt datasets, enabling a comparative analysis of energy use. Using the dataset, we investigate how prompt attributes, including length and complexity, correlate with energy expenditure. Our findings indicate substantial disparities in energy efficiency, suggesting ample scope for optimization and adoption of sustainable measures in LLM deployment. Our contribution lies not only in the MELODI framework but also in the novel dataset, a resource that can be expanded by other researchers. Thus, MELODI is a foundational tool and dataset for advancing research into energy-conscious LLM deployment, steering the field toward a more sustainable future., Comment: 11 pages, 5 figures. Submitted to NeurIPS 2024. The released code and dataset are available at https://github.com/ejhusom/MELODI

SpeechBrain is an open-source Conversational AI toolkit based on PyTorch, focused particularly on speech processing tasks such as speech recognition, speech enhancement, speaker recognition, text-to-speech, and much more. It promotes transparency and replicability by releasing both the pre-trained models and the complete "recipes" of code and algorithms required for training them. This paper presents SpeechBrain 1.0, a significant milestone in the evolution of the toolkit, which now has over 200 recipes for speech, audio, and language processing tasks, and more than 100 models available on Hugging Face. SpeechBrain 1.0 introduces new technologies to support diverse learning modalities, Large Language Model (LLM) integration, and advanced decoding strategies, along with novel models, tasks, and modalities. It also includes a new benchmark repository, offering researchers a unified platform for evaluating models across diverse tasks., Comment: Accepted to the Journal of Machine Learning research (JMLR), Machine Learning Open Source Software

Intellectual Property (IP) piracy, overbuilding, reverse engineering, and hardware Trojan are serious security concerns during integrated circuit (IC) development. Logic locking has proven to be a solid defence for mitigating these threats. The existing logic locking techniques are vulnerable to SAT-based attacks. However, several SAT-resistant logic locking methods are reported; they require significant overhead. This paper proposes a novel input dependent key-based logic locking (IDKLL) that effectively prevents SAT-based attacks with low overhead. We first introduce a novel idea of IDKLL, where a design is locked such that it functions correctly for all input patterns only when their corresponding valid key sequences are applied. In contrast to conventional logic locking, the proposed IDKLL method uses multiple key sequences (instead of a single key sequence) as a valid key that provides correct functionality for all inputs. Further, we propose a sub-circuit replacement based IDKLL approach called SubLock that locks the design by replacing the original sub-circuitry with the corresponding IDKLL based locked circuit to prevent SAT attack with low overhead. The experimental evaluation on ISCAS benchmarks shows that the proposed SubLock mitigates the SAT attack with high security and reduced overhead over the well-known existing methods., Comment: 22 pages, 12 figures, Journal

We employ a Bose Einstein Condensate (BEC) based atom-optic kicked rotor to generate an asymmetrically localized momentum distribution that depends upon initial velocity of the BEC. Asymmetric features are shown to arise from the early-time dynamics induced by the broken parity symmetry and, asymptotically freeze as the dynamical localization stabilizes. The asymmetry in the momentum distribution critically depends upon the initial launch velocity and is sensitive to very small initial velocities ('micromotion') of the BEC. In this work, we also perform a precise measurement of the 'micromotion'. By utilizing the technique of measuring the early-time asymmetry of momentum distribution, we report measurement of micromotion down to (230 \pm 17 , \mu\text{m/s}).

Matching a target spectrum with known spectra in a spectral library is a common method for material identification in hyperspectral imaging research. Hyperspectral spectra exhibit precise absorption features across different wavelength segments, and the unique shapes and positions of these absorptions create distinct spectral signatures for each material, aiding in their identification. Therefore, only the specific positions can be considered for material identification. This study introduces the Weighted Sum of Segmented Correlation method, which calculates correlation indices between various segments of a library and a test spectrum, and derives a matching index, favoring positive correlations and penalizing negative correlations using assigned weights. The effectiveness of this approach is evaluated for mineral identification in hyperspectral images from both Earth and Martian surfaces., Comment: Accepted in IEEE IGARSS 2024 conference

Low-rank model order reduction strategies for contact mechanics show limited dimensionality reduction due to linear inseparability of contact pressure field. Therefore, a dictionary based strategy is explored for creating efficient models for frictionless non-adhesive contact. A large dictionary of contact pressure trajectories is generated using a high-fidelity finite element model, while approximating the online query with a small number of dictionary entries. This is achieved by inducing sparsity in the approximation. Accuracy, computational effort and limitations of such methods are demonstrated on few numerical examples., Comment: arXiv admin note: substantial text overlap with arXiv:2405.20211

We report here our comparative analysis of active galactic nucleus (AGN) and star formation (SF) characteristics of a sample of narrow-line Seyfert 1 (NLS1) and broad-line Seyfert 1 (BLS1) galaxies. Our sample consisted of 373 BLS1 and 240 NLS1 galaxies spanning the redshift 0.02 to 0.8. The broad-band spectral energy distribution, constructed using data from the ultra-violet to the far-infrared, was modelled using CIGALE to derive the basic properties of our sample. We searched for differences in stellar mass, star formation rate (SFR), and AGN luminosity in the two populations. We also estimated new radiation-pressure corrected black hole masses for our sample of BLS1 and NLS1 galaxies. While the virial black hole mass (MBH) of BLS1 galaxies is similar to their radiation-pressure corrected MBH values, the virial MBH values of NLS1 galaxies are underestimated. We found that NLS1 galaxies have a lower MBH of log (MBH) = 7.45 +/- 0.27 and a higher log (Eddington ratio) of -0.72 +/- 0.22 than BLS1 galaxies, which have log (MBH) and log (Eddington ratio) of 8.04 +/- 0.26 and -1.08 +/- 0.24, respectively. The distributions of stellar mass, SFR, and specific star formation (sSFR = SFR/stellar mass) for the two populations are indistinguishable. This analysis is based on an independent approach and contradicts reports in the literature that NLS1 galaxies have a higher SF than BLS1 galaxies. While we found that AGN luminosity increases with stellar mass, luminosity of SF flattens at high stellar mass for both BLS1 and NLS1 galaxies. The reason may be that SF is suppressed by AGN feedback at stellar mass higher than 10^11 solar mass or that the AGN fuelling mechanism is decoupled from SF. Separating the sample into radio-detected and radio-undetected subsamples, we found no difference in their SF properties suggesting that the effect of AGN jets on SF is negligible., Comment: 15 pages, accepted for publication in A&A

In large deep neural networks that seem to perform surprisingly well on many tasks, we also observe a few failures related to accuracy, social biases, and alignment with human values, among others. Therefore, before deploying these models, it is crucial to characterize this failure landscape for engineers to debug and legislative bodies to audit models. Nevertheless, it is infeasible to exhaustively test for all possible combinations of factors that could lead to a model's failure. In this paper, we introduce a post-hoc method that utilizes \emph{deep reinforcement learning} to explore and construct the landscape of failure modes in pre-trained discriminative and generative models. With the aid of limited human feedback, we then demonstrate how to restructure the failure landscape to be more desirable by moving away from the discovered failure modes. We empirically show the effectiveness of the proposed method across common Computer Vision, Natural Language Processing, and Vision-Language tasks., Comment: 25 pages, 35 figures

We present a logic based interpretable model for learning on graphs and an algorithm to distill this model from a Graph Neural Network (GNN). Recent results have shown connections between the expressivity of GNNs and the two-variable fragment of first-order logic with counting quantifiers (C2). We introduce a decision-tree based model which leverages an extension of C2 to distill interpretable logical classifiers from GNNs. We test our approach on multiple GNN architectures. The distilled models are interpretable, succinct, and attain similar accuracy to the underlying GNN. Furthermore, when the ground truth is expressible in C2, our approach outperforms the GNN., Comment: To Appear in the Proceedings of KR 2024

Spoken Language Understanding (SLU) is a critical component of voice assistants; it consists of converting speech to semantic parses for task execution. Previous works have explored end-to-end models to improve the quality and robustness of SLU models with Deliberation, however these models have remained autoregressive, resulting in higher latencies. In this work we introduce PRoDeliberation, a novel method leveraging a Connectionist Temporal Classification-based decoding strategy as well as a denoising objective to train robust non-autoregressive deliberation models. We show that PRoDeliberation achieves the latency reduction of parallel decoding (2-10x improvement over autoregressive models) while retaining the ability to correct Automatic Speech Recognition (ASR) mistranscriptions of autoregressive deliberation systems. We further show that the design of the denoising training allows PRoDeliberation to overcome the limitations of small ASR devices, and we provide analysis on the necessity of each component of the system.

Estimating the correlation coefficient has been a daunting work with the increasing complexity of dataset's pattern. One of the problems in manufacturing applications consists of the estimation of a critical process variable during a machining operation from directly measurable process variables. For example, the prediction of surface roughness of a workpiece during finish turning processes. In this paper, we did exhaustive study on the existing popular correlation coefficients: Pearson correlation coefficient, Spearman's rank correlation coefficient, Kendall's Tau correlation coefficient, Fechner correlation coefficient, and Nonlinear correlation coefficient. However, no one of them can capture all the nonlinear and linear correlations. So, we represent a universal non-linear non-parametric correlation measurement, g-correlation coefficient. Unlike other correlation measurements, g-correlation doesn't require assumptions and pick the dominating patterns of the dataset after examining all the major patterns no matter it is linear or nonlinear. Results of testing on both linearly correlated and non-linearly correlated dataset and comparison with the introduced correlation coefficients in literature show that g-correlation is robust on all the linearly correlated dataset and outperforms for some non-linearly correlated dataset. Results of the application of different correlation concepts to surface roughness assessment show that g-correlation has a central role among all standard concepts of correlation.

INTRODUCTION: The pharmacological treatment of Major Depressive Disorder (MDD) relies on a trial-and-error approach. We introduce an artificial intelligence (AI) model aiming to personalize treatment and improve outcomes, which was deployed in the Artificial Intelligence in Depression Medication Enhancement (AIDME) Study. OBJECTIVES: 1) Develop a model capable of predicting probabilities of remission across multiple pharmacological treatments for adults with at least moderate major depression. 2) Validate model predictions and examine them for amplification of harmful biases. METHODS: Data from previous clinical trials of antidepressant medications were standardized into a common framework and included 9,042 adults with moderate to severe major depression. Feature selection retained 25 clinical and demographic variables. Using Bayesian optimization, a deep learning model was trained on the training set, refined using the validation set, and tested once on the held-out test set. RESULTS: In the evaluation on the held-out test set, the model demonstrated achieved an AUC of 0.65. The model outperformed a null model on the test set (p = 0.01). The model demonstrated clinical utility, achieving an absolute improvement in population remission rate in hypothetical and actual improvement testing. While the model did identify one drug (escitalopram) as generally outperforming the other drugs (consistent with the input data), there was otherwise significant variation in drug rankings. On bias testing, the model did not amplify potentially harmful biases. CONCLUSIONS: We demonstrate the first model capable of predicting outcomes for 10 different treatment options for patients with MDD, intended to be used at or near the start of treatment to personalize treatment. The model was put into clinical practice during the AIDME randomized controlled trial whose results are reported separately.

Diffusion-based image generation models have achieved great success in recent years by showing the capability of synthesizing high-quality content. However, these models contain a huge number of parameters, resulting in a significantly large model size. Saving and transferring them is a major bottleneck for various applications, especially those running on resource-constrained devices. In this work, we develop a novel weight quantization method that quantizes the UNet from Stable Diffusion v1.5 to 1.99 bits, achieving a model with 7.9X smaller size while exhibiting even better generation quality than the original one. Our approach includes several novel techniques, such as assigning optimal bits to each layer, initializing the quantized model for better performance, and improving the training strategy to dramatically reduce quantization error. Furthermore, we extensively evaluate our quantized model across various benchmark datasets and through human evaluation to demonstrate its superior generation quality., Comment: NeurIPS 2024. Project Page: https://snap-research.github.io/BitsFusion

Evolutionary pathways of binary systems are vastly different from single stellar evolution, and thus, there is a need to quantify their frequency and diversity. Open clusters are the best test-bed to unveil the secrets of binary populations due to their coeval nature. And the availability of multi-wavelength data in recent years has been critical in characterising the binary population. NGC 752 is a solar metallicity, intermediate-age open cluster located at 460 pc. In this work, we aim to identify the optically subluminous white dwarfs in NGC 752 and identify the illusive blue lurkers by association. We used multiwavelength photometry from Astrosat/UVIT, swift/UVOT, Gaia DR3 and other archival surveys to analyse the colour-magnitude diagrams and spectral energy distributions of 37 cluster members. We detected eight white dwarfs as companions to cluster members. Four of the systems are main sequence stars with extremely low mass white dwarfs as their companions. Two are these main sequence stars are also fast rotators. The presence of low mass white dwarfs and high rotation signals a past mass transfer, and we classified the four main sequence stars as blue lurkers. The binary fraction in NGC 752 was estimated to be 50--70%, and it shows that the contribution of optically undetected stars is crucial in quantifying the present-day binary fraction., Comment: 7 pages, 3 figures, accepted in Astronomy & Astrophysics

On-device training is currently the most common approach for training machine learning (ML) models on private, distributed user data. Despite this, on-device training has several drawbacks: (1) most user devices are too small to train large models on-device, (2) on-device training is communication- and computation-intensive, and (3) on-device training can be difficult to debug and deploy. To address these problems, we propose Private Evolution-Text (PrE-Text), a method for generating differentially private (DP) synthetic textual data. First, we show that across multiple datasets, training small models (models that fit on user devices) with PrE-Text synthetic data outperforms small models trained on-device under practical privacy regimes ($\epsilon=1.29$, $\epsilon=7.58$). We achieve these results while using 9$\times$ fewer rounds, 6$\times$ less client computation per round, and 100$\times$ less communication per round. Second, finetuning large models on PrE-Text's DP synthetic data improves large language model (LLM) performance on private data across the same range of privacy budgets. Altogether, these results suggest that training on DP synthetic data can be a better option than training a model on-device on private distributed data. Code is available at https://github.com/houcharlie/PrE-Text., Comment: ICML 2024 (Oral). Latest revision corrects a discussion on concurrent work arXiv:2403.01749. We described their work as reliant on using closed-sourced models when in reality they also evaluate and use open source models. This has been corrected in this version