Search

Instruction finetuning is a popular paradigm to align large language models (LLM) with human intent. Despite its popularity, this idea is less explored in improving the LLMs to align existing foundation models with scientific disciplines, concepts and goals. In this work, we present SciTune as a tuning framework to improve the ability of LLMs to follow scientific multimodal instructions. To test our methodology, we use a human-generated scientific instruction tuning dataset and train a large multimodal model LLaMA-SciTune that connects a vision encoder and LLM for science-focused visual and language understanding. In comparison to the models that are finetuned with machine generated data only, LLaMA-SciTune surpasses human performance on average and in many sub-categories on the ScienceQA benchmark., Comment: Preprint. Work in progress

Uncertainty quantification is a critical yet unsolved challenge for deep learning, especially for the time series imputation with irregularly sampled measurements. To tackle this problem, we propose a novel framework based on the principles of recurrent neural networks and neural stochastic differential equations for reconciling irregularly sampled measurements. We impute measurements at any arbitrary timescale and quantify the uncertainty in the imputations in a principled manner. Specifically, we derive analytical expressions for quantifying and propagating the epistemic and aleatoric uncertainty across time instants. Our experiments on the IEEE 37 bus test distribution system reveal that our framework can outperform state-of-the-art uncertainty quantification approaches for time-series data imputations., Comment: 7 pages, 3 figures

We study the problem of learning features through self-supervision that are generalisable to multiple graphs. State-of-the-art graph self-supervision restricts training to only one graph, resulting in graph-specific models that are incompatible with different but related graphs. We hypothesize that training with more than one graph that belong to the same family can improve the quality of the learnt representations. However, learning universal features from disparate node/edge features in different graphs is non-trivial. To address this challenge, we first homogenise the disparate features with graph-specific encoders that transform the features into a common space. A universal representation learning module then learns generalisable features on this common space. We show that compared to traditional self-supervision with one graph, our approach results in (1) better performance on downstream node classification, (2) learning features that can be re-used for unseen graphs of the same family, (3) more efficient training and (4) compact yet generalisable models. We also show ability of the proposed framework to deliver these benefits for relatively larger graphs. In this paper, we present a principled way to design foundation graph models that learn from more than one graph in an end-to-end manner, while bridging the gap between self-supervised and supervised performance., Comment: arXiv admin note: text overlap with arXiv:2302.11939, arXiv:2301.13287, arXiv:2305.12686, arXiv:2305.02299

Deep reinforcement learning (DRL) has empowered a variety of artificial intelligence fields, including pattern recognition, robotics, recommendation-systems, and gaming. Similarly, graph neural networks (GNN) have also demonstrated their superior performance in supervised learning for graph-structured data. In recent times, the fusion of GNN with DRL for graph-structured environments has attracted a lot of attention. This paper provides a comprehensive review of these hybrid works. These works can be classified into two categories: (1) algorithmic enhancement, where DRL and GNN complement each other for better utility; (2) application-specific enhancement, where DRL and GNN support each other. This fusion effectively addresses various complex problems in engineering and life sciences. Based on the review, we further analyze the applicability and benefits of fusing these two domains, especially in terms of increasing generalizability and reducing computational complexity. Finally, the key challenges in integrating DRL and GNN, and potential future research directions are highlighted, which will be of interest to the broader machine learning community., Comment: 20 pages, 3 figures, 2 tables

Influence maximization (IM) is a combinatorial problem of identifying a subset of nodes called the seed nodes in a network (graph), which when activated, provide a maximal spread of influence in the network for a given diffusion model and a budget for seed set size. IM has numerous applications such as viral marketing, epidemic control, sensor placement and other network-related tasks. However, the uses are limited due to the computational complexity of current algorithms. Recently, learning heuristics for IM have been explored to ease the computational burden. However, there are serious limitations in current approaches such as: (1) IM formulations only consider influence via spread and ignore self activation; (2) scalability to large graphs; (3) generalizability across graph families; (4) low computational efficiency with a large running time to identify seed sets for every test network. In this work, we address each of these limitations through a unique approach that involves (1) formulating a generic IM problem as a Markov decision process that handles both intrinsic and influence activations; (2) employing double Q learning to estimate seed nodes; (3) ensuring scalability via sub-graph based representations; and (4) incorporating generalizability via meta-learning across graph families. Extensive experiments are carried out in various standard networks to validate performance of the proposed Graph Meta Reinforcement learning (GraMeR) framework. The results indicate that GraMeR is multiple orders faster and generic than conventional approaches., Comment: 11 pages, 6 figures

Graph Neural Networks (GNN) provide a powerful framework that elegantly integrates Graph theory with Machine learning for modeling and analysis of networked data. We consider the problem of quantifying the uncertainty in predictions of GNN stemming from modeling errors and measurement uncertainty. We consider aleatoric uncertainty in the form of probabilistic links and noise in feature vector of nodes, while epistemic uncertainty is incorporated via a probability distribution over the model parameters. We propose a unified approach to treat both sources of uncertainty in a Bayesian framework, where Assumed Density Filtering is used to quantify aleatoric uncertainty and Monte Carlo dropout captures uncertainty in model parameters. Finally, the two sources of uncertainty are aggregated to estimate the total uncertainty in predictions of a GNN. Results in the real-world datasets demonstrate that the Bayesian model performs at par with a frequentist model and provides additional information about predictions uncertainty that are sensitive to uncertainties in the data and model., Comment: 10 pages, 1 figure, 6 Tables

Identifying critical nodes and links in graphs is a crucial task. These nodes/links typically represent critical elements/communication links that play a key role in a system's performance. However, a majority of the methods available in the literature on the identification of critical nodes/links are based on an iterative approach that explores each node/link of a graph at a time, repeating for all nodes/links in the graph. Such methods suffer from high computational complexity and the resulting analysis is also network-specific. To overcome these challenges, this article proposes a scalable and generic graph neural network (GNN) based framework for identifying critical nodes/links in large complex networks. The proposed framework defines a GNN based model that learns the node/link criticality score on a small representative subset of nodes/links. An appropriately trained model can be employed to predict the scores of unseen nodes/links in large graphs and consequently identify the most critical ones. The scalability of the framework is demonstrated through prediction of nodes/links scores in large scale synthetic and real-world networks. The proposed approach is fairly accurate in approximating the criticality scores and offers a significant computational advantage over conventional approaches., Comment: 29 pages, single column, 3 figures

In the quest to improve efficiency, interdependence and complexity are becoming defining characteristics of modern complex networks representing engineered and natural systems. Graph theory is a widely used framework for modeling such complex networks and to evaluate their robustness to disruptions. Particularly, identification of critical nodes/links in a graph can facilitate the enhancement of graph (system) robustness and characterize crucial factors of system performance. Most existing methods of critical node identification are based on an iterative approach that explores each node/link of a graph. These methods suffer from high computational complexity and the resulting analysis is network specific. Additionally, uncertainty associated with the underlying graphical model further limits the potential value of these traditional approaches. To overcome these challenges, we propose a Bayesian graph neural network based node classification framework that is computationally efficient and systematically incorporates uncertainties. Instead of utilizing the observed graph for training the model, a MAP estimate of the graph is computed based on the observed topology and node target labels. Further, a Monte-Carlo (MC) dropout algorithm is incorporated to account for the epistemic uncertainty. The fidelity and the gain in computational complexity offered by the Bayesian framework is illustrated using simulation results., Comment: 6 pages, 2 figures, 3 Tables