Search

Large Language Models (LLMs) excel in diverse applications including generation of code snippets, but often struggle with generating code for complex Machine Learning (ML) tasks. Although existing LLM single-agent based systems give varying performance depending on the task complexity, they purely rely on larger and expensive models such as GPT-4. Our investigation reveals that no-cost and low-cost models such as Gemini-Pro, Mixtral and CodeLlama perform far worse than GPT-4 in a single-agent setting. With the motivation of developing a cost-efficient LLM based solution for solving ML tasks, we propose an LLM Multi-Agent based system which leverages combination of experts using profiling, efficient retrieval of past observations, LLM cascades, and ask-the-expert calls. Through empirical analysis on ML engineering tasks in the MLAgentBench benchmark, we demonstrate the effectiveness of our system, using no-cost models, namely Gemini as the base LLM, paired with GPT-4 in cascade and expert to serve occasional ask-the-expert calls for planning. With 94.2\% reduction in the cost (from \$0.931 per run cost averaged over all tasks for GPT-4 single agent system to \$0.054), our system is able to yield better average success rate of 32.95\% as compared to GPT-4 single-agent system yielding 22.72\% success rate averaged over all the tasks of MLAgentBench., Comment: Presented at AIMLSystems '24

Large Language Models (LLMs) often exhibit positional bias in long-context settings, under-attending to information in the middle of inputs. We investigate the presence of this bias in long-form summarization, its impact on faithfulness, and various techniques to mitigate this bias. To consistently evaluate faithfulness, we first compile a benchmark of eight human-annotated long-form summarization datasets and perform a meta-evaluation of faithfulness metrics. We show that LLM-based faithfulness metrics, though effective with full-context inputs, remain sensitive to document order, indicating positional bias. Analyzing LLM-generated summaries across six datasets, we find a "U-shaped" trend in faithfulness, where LLMs faithfully summarize the beginning and end of documents but neglect middle content. Perturbing document order similarly reveals models are less faithful when important documents are placed in the middle of the input. We find that this behavior is partly due to shifting focus with context length: as context increases, summaries become less faithful, but beyond a certain length, faithfulness improves as the model focuses on the end. Finally, we experiment with different generation techniques to reduce positional bias and find that prompting techniques effectively direct model attention to specific positions, whereas more sophisticated approaches offer limited improvements. Our data and code are available in https://github.com/meetdavidwan/longformfact., Comment: 18 pages

Session-based recommendation (SR) models aim to recommend top-K items to a user, based on the user's behaviour during the current session. Several SR models are proposed in the literature, however,concerns have been raised about their susceptibility to inherent biases in the training data (observed data) such as popularity bias. SR models when trained on the biased training data may encounter performance challenges on out-of-distribution data in real-world scenarios. One way to mitigate popularity bias is counterfactual data augmentation. Compared to prior works that rely on generating data using SR models, we focus on utilizing the capabilities of state-of-the art diffusion models for generating counterfactual data. We propose a guided diffusion-based counterfactual augmentation framework for SR. Through a combination of offline and online experiments on a real-world and simulated dataset, respectively, we show that our approach performs significantly better than the baseline SR models and other state-of-the art augmentation frameworks. More importantly, our framework shows significant improvement on less popular target items, by achieving up to 20% gain in Recall and 13% gain in CTR on real-world and simulated datasets,respectively.

Symplectic billiards were introduced by Albers and Tabachnikov as billiards in strictly convex bounded domains of the plane with smooth boundary having a specific law of reflection. This paper proves a rigidity result for symplectic billiards which is similar to a previous result on classical billiards formulated by De Simoi, Kaloshin and Wei. Namely, it states that close to an ellipse, a sufficiently smooth one-parameter family of axially symmetric domains either contains domains with different area-spectra or is trivial, in a sense that the domains differ by area-preserving affine transformations of the plane. The paper also prove that in the general setting - that is even if the domains are not close to an ellipse - any sufficiently smooth one-parameter family of axially symmetric domains which preserves the area-spectrum is tangent to a finite dimensionnal space., Comment: 29 pages, 2 figures

This paper shows that the wave trace of a bounded and strictly convex planar domain may be arbitrarily smooth in a neighborhood of some point in the length spectrum. In other words, the Poisson relation, which asserts that the singular support of the wave trace is contained in the closure of $\pm$ the length spectrum, can almost be made into a strict inclusion. To do so, we construct large families of domains for which there exist multiple periodic billiard orbits having the same length but different Maslov indices. Using the microlocal Balian-Bloch-Zelditch parametrix for wave invariants developed in our previous paper, we solve a large system of equations for the boundary curvature jets, which leads to the required cancellations. We call such periodic orbits silent, since they are undetectable from the ostensibly audible wave trace. Such cancellations show that there are potential limitations in using the wave trace for inverse spectral problems and more fundamentally, that the Laplace spectrum and length spectrum are inherently different mathematical objects, at least insofar as the wave trace is concerned., Comment: 61 pages, 8 figures

LLMs are increasingly being deployed for multilingual applications and have demonstrated impressive translation capabilities between several low and high-resource languages. An aspect of translation that often gets overlooked is that of cultural adaptation, or modifying source culture references to suit the target culture. While specialized translation models still outperform LLMs on the machine translation task when viewed from the lens of correctness, they are not sensitive to cultural differences often requiring manual correction. LLMs on the other hand have a rich reservoir of cultural knowledge embedded within its parameters that can be potentially exploited for such applications. In this paper, we define the task of cultural adaptation and create an evaluation framework to evaluate the performance of modern LLMs for cultural adaptation and analyze their cross-cultural knowledge while connecting related concepts across different cultures. We also analyze possible issues with automatic adaptation. We hope that this task will offer more insight into the cultural understanding of LLMs and their creativity in cross-cultural scenarios., Comment: Accepted to CoNLL 2024

Low radio frequency spectral index measurements are a powerful tool to distinguish between different emission mechanisms and, in turn, to understand the nature of the sources. Besides the standard method of estimating the ``broadband" spectral index of sources from observations in two different frequency ``bands", if the observations were made with large instantaneous bandwidth, the ``in-band" spectral index can be determined, either using images of emission at multiple frequency ranges within a band or using the novel Multi Term-Multi Frequency Synthesis (MT-MFS) imaging algorithm. Here, using simulated upgraded Giant Metrewave Radio Telescope (uGMRT) data, we have systematically studied the reliability of various methods of spectral index estimation for sources with a wide range of signal-to-noise ratio (SNR). It is found that, for synthetic uGMRT point source data, the MT-MFS imaging algorithm produces in-band spectral indices for SNR~$\lesssim100$ that have errors $\gtrsim 0.2$, making them unreliable. However, at a similar SNR, the sub-band splitting method produces errors $\lesssim 0.2$, which are more accurate and unbiased in-band spectral indices. The broadband spectral indices produce errors $\lesssim 0.2$ even for SNR $\gtrsim 15$, and hence, they are most reliable if there are no higher-order variations in the spectral index. These results may be used to improve the uGMRT observation and data analysis strategies depending on the brightness of the target source., Comment: 10 pages, 5 figures, accepted by ApJ on 24th May 2024

Robotic Process Automation (RPA) systems face challenges in handling complex processes and diverse screen layouts that require advanced human-like decision-making capabilities. These systems typically rely on pixel-level encoding through drag-and-drop or automation frameworks such as Selenium to create navigation workflows, rather than visual understanding of screen elements. In this context, we present SmartFlow, an AI-based RPA system that uses pre-trained large language models (LLMs) coupled with deep-learning based image understanding. Our system can adapt to new scenarios, including changes in the user interface and variations in input data, without the need for human intervention. SmartFlow uses computer vision and natural language processing to perceive visible elements on the graphical user interface (GUI) and convert them into a textual representation. This information is then utilized by LLMs to generate a sequence of actions that are executed by a scripting engine to complete an assigned task. To assess the effectiveness of SmartFlow, we have developed a dataset that includes a set of generic enterprise applications with diverse layouts, which we are releasing for research use. Our evaluations on this dataset demonstrate that SmartFlow exhibits robustness across different layouts and applications. SmartFlow can automate a wide range of business processes such as form filling, customer service, invoice processing, and back-office operations. SmartFlow can thus assist organizations in enhancing productivity by automating an even larger fraction of screen-based workflows. The demo-video and dataset are available at https://smartflow-4c5a0a.webflow.io/., Comment: 32nd ACM International Conference on Information and Knowledge Management

Creative story illustration requires a consistent interplay of multiple characters or objects. However, conventional text-to-image models face significant challenges while producing images featuring multiple personalized subjects. For example, they distort the subject rendering, or the text descriptions fail to render coherent subject interactions. We present Multi-Subject Personalization (MSP) to alleviate some of these challenges. We implement MSP using Stable Diffusion and assess our approach against other text-to-image models, showcasing its consistent generation of good-quality images representing intended subjects and interactions., Comment: 2023 Conference on Neural Information Processing Systems

Textual image generation spans diverse fields like advertising, education, product packaging, social media, information visualization, and branding. Despite recent strides in language-guided image synthesis using diffusion models, current models excel in image generation but struggle with accurate text rendering and offer limited control over font attributes. In this paper, we aim to enhance the synthesis of high-quality images with precise text customization, thereby contributing to the advancement of image generation models. We call our proposed method CustomText. Our implementation leverages a pre-trained TextDiffuser model to enable control over font color, background, and types. Additionally, to address the challenge of accurately rendering small-sized fonts, we train the ControlNet model for a consistency decoder, significantly enhancing text-generation performance. We assess the performance of CustomText in comparison to previous methods of textual image generation on the publicly available CTW-1500 dataset and a self-curated dataset for small-text generation, showcasing superior results., Comment: Accepted by AI for Content Creation (AI4CC) workshop at CVPR 2024

This paper is part I of a series in which we aim to show that the singular support of the wave trace and the length spectrum of a smooth, strictly convex, and bounded planar billiard table are generally distinct objects. We derive an asymptotic trace formula for the regularized resolvent which is dual to the wave trace and contains the same information. To do this, we consider a class of periodic orbits which have nearly degenerate Poincar\'e maps and study their leading order behavior as the deformation parameter goes to zero, generating large coefficients in the wave trace. We also keep careful track of the Maslov indices, which will allow us to match contributions of opposite signs in our subsequent paper. Each cancellation of coefficients in the resolvent trace corresponds to making the wave trace one degree smoother. The resolvent based approach is due to Balian and Bloch and was significantly expanded upon by Zelditch in a foundational series of papers [Zel09], [Zel04a], [Zel04c] and [Zel00]., Comment: 28 pages, 1 figure

The paper demonstrates the spectroscopic and photometric capabilities of the Ultra-Violet Imaging Telescope (UVIT) to study T-Tauri stars (TTSs). We present the first UVIT/Far-UV spectrum of a TTS, TW Hya. Based on C IV line luminosity, we estimated accretion luminosity (0.1 $L_\odot$) and mass accretion rate (2.2 $\times$ $10^{-8} M_\odot /yr$) of TW Hya, and compared these values with the accretion luminosity (0.03 $L_\odot$) and mass accretion rate (0.6 $\times$ $10^{-8} M_\odot /yr$) derived from spectral energy distribution (SED). From the SED, we derive best-fitted parameters for TW Hya: $T_{eff}$ = 3900$\pm$50 K, radius = 1.2$\pm$0.03 $R_\odot$, $\mathrm{log}\, g = 4.0$ and equivalent black-body temperatures corresponding to accretion luminosity as 14100$\pm$25 K. The parameters of TW Hya derived from UVIT observations were found to be matched well with the literature. Comparison with IUE spectra also suggests that UVIT can be used to study the spectroscopic variability of young stars. This study proposes leveraging the FUV spectroscopic capabilities of UVIT to contribute to the advancement of upcoming UV spectroscopic missions, including the Indian Spectroscopic Imaging Space Telescope (INSIST)., Comment: 9 pages, 3 figures, 2 tables

Several tools have recently been proposed for assisting researchers during various stages of the research life-cycle. However, these primarily concentrate on tasks such as retrieving and recommending relevant literature, reviewing and critiquing the draft, and writing of research manuscripts. Our investigation reveals a significant gap in availability of tools specifically designed to assist researchers during the challenging ideation phase of the research life-cycle. To aid with research ideation, we propose `Acceleron', a research accelerator for different phases of the research life cycle, and which is specially designed to aid the ideation process. Acceleron guides researchers through the formulation of a comprehensive research proposal, encompassing a novel research problem. The proposals motivation is validated for novelty by identifying gaps in the existing literature and suggesting a plausible list of techniques to solve the proposed problem. We leverage the reasoning and domain-specific skills of Large Language Models (LLMs) to create an agent-based architecture incorporating colleague and mentor personas for LLMs. The LLM agents emulate the ideation process undertaken by researchers, engaging researchers in an interactive fashion to aid in the development of the research proposal. Notably, our tool addresses challenges inherent in LLMs, such as hallucinations, implements a two-stage aspect-based retrieval to manage precision-recall trade-offs, and tackles issues of unanswerability. As evaluation, we illustrate the execution of our motivation validation and method synthesis workflows on proposals from the ML and NLP domain, given by 3 distinct researchers. Our observations and evaluations provided by the researchers illustrate the efficacy of the tool in terms of assisting researchers with appropriate inputs at distinct stages and thus leading to improved time efficiency., Comment: Accepted at AI2ASE Workshop at AAAI'24 Conference. 13 Pages and 4 Figures

Self-training techniques have shown remarkable value across many deep learning models and tasks. However, such techniques remain largely unexplored when considered in the context of learning fast solvers for systems of partial differential equations (Eg: Neural Operators). In this work, we explore the use of self-training for Fourier Neural Operators (FNO). Neural Operators emerged as a data driven technique, however, data from experiments or traditional solvers is not always readily available. Physics Informed Neural Operators (PINO) overcome this constraint by utilizing a physics loss for the training, however the accuracy of PINO trained without data does not match the performance obtained by training with data. In this work we show that self-training can be used to close this gap in performance. We examine canonical examples, namely the 1D-Burgers and 2D-Darcy PDEs, to showcase the efficacy of self-training. Specifically, FNOs, when trained exclusively with physics loss through self-training, approach 1.07x for Burgers and 1.02x for Darcy, compared to FNOs trained with both data and physics loss. Furthermore, we discover that pseudo-labels can be used for self-training without necessarily training to convergence in each iteration. A consequence of this is that we are able to discover self-training schedules that improve upon the baseline performance of PINO in terms of accuracy as well as time., Comment: Paper accepted as a Spotlight talk at Symbiosis of Deep Learning and Differential Equations, Neural Information Processing Systems 2023

We present a multiwavelength investigation of the hub-filament system RCW 117 (IRAS 17059-4132), which shows intricate filamentary features in the far-infrared, mapped using Herschel images. We obtain the column density and dust temperature maps for the region using the Herschel images, and identify 88 cores and 12 filaments from the column density map of the region ($18'\times18'$). We employ the ThrUMMS $^{13}$CO (J=1-0) data for probing the kinematics in RCW 117, and find velocity gradients ($\sim 0.3-1$ km s$^{-1}$ pc$^{-1}$) with hints of matter inflow along the filamentary structures. Ionised gas emission from the associated HII region is examined using the Giant Metrewave Radio Telescope (GMRT) at 610 and 1280 MHz, and is found to be of extent $5 \times 3$ pc$^2$ with intensity being brightest towards the hub. We estimate the peak electron density towards the hub to be $\sim 750$ cm$^{-3}$. Thirty four Class 0/I young stellar objects (YSOs) have been identified in the region using the Spitzer GLIMPSE colour-colour diagram, with many lying along the filamentary structures. Based on the (i) presence of filamentary structures, (ii) distribution of cores across the region, with $\sim39$% found along the filamentary structures, (iii) massive star-formation tracers in the hub, and (iv) the kinematics, we believe that global hierarchical collapse can plausibly explain the observed features in RCW 117., Comment: 15 pages, 11 figures, 6 tables, Accepted for publication in Monthly Notices of the Royal Astronomical Society (MNRAS)

We prove that equivalence classes of marked length isospectral Birkhoff billiard tables are compact in the $C^\infty$ topology, analogous to the Laplace spectral results of Melrose, Osgood, Phillips and Sarnak. To do so, we derive a hierarchical structure for the integral invariants of Marvizi and Melrose, or equivalently the coefficients of a caustic length-Lazutkin parameter expansion, which are in turn algebraically equivalent to the Taylor coefficients of Mather's $\beta$ function (also called the mean minimal action). Under a generically satisfied noncoincidence condition, these are also Laplace spectral invariants and can be used to hear the shape of certain drumheads. As a byproduct, we obtain an independent proof of the compactness of Laplace isospectral sets for strictly convex planar billiard tables. The proof of the structure theorem uses an interpolating Hamiltonian for nearly glancing billiard orbits and some analytic number theory to compute its Taylor coefficients., Comment: 70 pages, 4 figures. Compactness upgraded to the smooth category

Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing. However, standard chat-based conversational interfaces do not support transparency and verifiability of the editing changes that they suggest. To give the author more agency when editing with an LLM, we present InkSync, an editing interface that suggests executable edits directly within the document being edited. Because LLMs are known to introduce factual errors, Inksync also supports a 3-stage approach to mitigate this risk: Warn authors when a suggested edit introduces new information, help authors Verify the new information's accuracy through external search, and allow an auditor to perform an a-posteriori verification by Auditing the document via a trace of all auto-generated content. Two usability studies confirm the effectiveness of InkSync's components when compared to standard LLM-based chat interfaces, leading to more accurate, more efficient editing, and improved user experience.

We apply machine learning algorithms to classify Infrared (IR)-selected targets for NASA's upcoming SPHEREx mission. In particular, we are interested in classifying Young Stellar Objects (YSOs), which are essential for understanding the star formation process. Our approach differs from previous work, which has relied heavily on broadband color criteria to classify IR-bright objects, and are typically implemented in color-color and color-magnitude diagrams. However, these methods do not state the confidence associated with the classification and the results from these methods are quite ambiguous due to the overlap of different source types in these diagrams. Here, we utilize photometric colors and magnitudes from seven near and mid-infrared bands simultaneously and employ machine and deep learning algorithms to carry out probabilistic classification of YSOs, Asymptotic Giant Branch (AGB) stars, Active Galactic Nuclei (AGN) and main-sequence (MS) stars. Our approach also sub-classifies YSOs into Class I, II, III and flat spectrum YSOs, and AGB stars into carbon-rich and oxygen-rich AGB stars. We apply our methods to infrared-selected targets compiled in preparation for SPHEREx which are likely to include YSOs and other classes of objects. Our classification indicates that out of $8,308,384$ sources, $1,966,340$ have class prediction with probability exceeding $90\%$, amongst which $\sim 1.7\%$ are YSOs, $\sim 58.2\%$ are AGB stars, $\sim 40\%$ are (reddened) MS stars, and $\sim 0.1\%$ are AGN whose red broadband colors mimic YSOs. We validate our classification using the spatial distributions of predicted YSOs towards the Cygnus-X star-forming complex, as well as AGB stars across the Galactic plane., Comment: 17 pages, 12 figures, Accepted for publication in MNRAS

Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LLMs remain confined behind proprietary walls, hindering scientific progress. Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context. To address this, we have trained XGen, a series of 7B parameter models on up to 8K sequence length for up to 1.5T tokens. We have also finetuned the XGen models on public-domain instructional data, creating their instruction-tuned counterparts (XGen-Inst). We open-source our models for both research advancements and commercial applications. Our evaluation on standard benchmarks shows that XGen models achieve comparable or better results when compared with state-of-the-art open-source LLMs. Our targeted evaluation on long sequence modeling tasks shows the benefits of our 8K-sequence models over 2K-sequence open-source LLMs.

Simulating physical systems using Partial Differential Equations (PDEs) has become an indispensible part of modern industrial process optimization. Traditionally, numerical solvers have been used to solve the associated PDEs, however recently Transform-based Neural Operators such as the Fourier Neural Operator and Wavelet Neural Operator have received a lot of attention for their potential to provide fast solutions for systems of PDEs. In this work, we investigate the importance of the transform layers to the reported success of transform based neural operators. In particular, we record the cost in terms of performance, if all the transform layers are replaced by learnable linear layers. Surprisingly, we observe that linear layers suffice to provide performance comparable to the best-known transform-based layers and seem to do so with a compute time advantage as well. We believe that this observation can have significant implications for future work on Neural Operators, and might point to other sources of efficiencies for these architectures., Comment: 8 pages, 3 figures, 4 tables

Physics-informed neural networks (PINNs) have been widely used to develop neural surrogates for solutions of Partial Differential Equations. A drawback of PINNs is that they have to be retrained with every change in initial-boundary conditions and PDE coefficients. The Hypernetwork, a model-based meta learning technique, takes in a parameterized task embedding as input and predicts the weights of PINN as output. Predicting weights of a neural network however, is a high-dimensional regression problem, and hypernetworks perform sub-optimally while predicting parameters for large base networks. To circumvent this issue, we use a low ranked adaptation (LoRA) formulation to decompose every layer of the base network into low-ranked tensors and use hypernetworks to predict the low-ranked tensors. Despite the reduced dimensionality of the resulting weight-regression problem, LoRA-based Hypernetworks violate the underlying physics of the given task. We demonstrate that the generalization capabilities of LoRA-based hypernetworks drastically improve when trained with an additional physics-informed loss component (HyperPINN) to satisfy the governing differential equations. We observe that LoRA-based HyperPINN training allows us to learn fast solutions for parameterized PDEs like Burger's equation and Navier Stokes: Kovasznay flow, while having an 8x reduction in prediction parameters on average without compromising on accuracy when compared to all other baselines., Comment: 8 pages, 4 figures, 3 Tables

Context. The advent of wide-band receiver systems on interferometer arrays enables one to undertake high-sensitivity and high-resolution radio continuum surveys of the Galactic plane in a reasonable amount of telescope time. However, to date, there are only a few such studies of the first quadrant of the Milky Way that have been carried out at frequencies below 1 GHz. The Giant Metrewave Radio Telescope (GMRT) has recently upgraded its receivers with wide-band capabilities (now called the uGMRT) and provides a good opportunity to conduct high resolution surveys, while also being sensitive to the extended structures. Aims. We wish to assess the feasibility of conducting a large-scale snapshot survey, the Metrewave Galactic Plane with the uGMRT Survey (MeGaPluG), to simultaneously map extended sources and compact objects at an angular resolution lower than $10''$ and a point source sensitivity of 0.15 mJy/beam. Methods. We performed an unbiased survey of a small portion of the Galactic plane, covering the W43/W44 regions ($l=29^\circ-35^\circ$ and $|b|<1^\circ$) in two frequency bands: 300$-$500 MHz and 550$-$750 MHz. The 200 MHz wide-band receivers on the uGMRT are employed to observe the target field in several pointings, spending nearly 14 minutes on each pointing in two separate scans. We developed an automated pipeline for the calibration, and a semi-automated self-calibration procedure is used to image each pointing using multi-scale CLEAN and outlier fields. Results. We produced continuum mosaics of the surveyed region at a final common resolution of $25''$ in the two bands that have central frequencies of 400 MHz and 650 MHz, with a point source sensitivity better than 5 mJy/beam. We plan to cover a larger footprint of the Galactic plane in the near future based on the lessons learnt from this study. (Abridged), Comment: To be published in A&A. 13 pages, 10 figures

RAFGL2591 is a massive star-forming complex in the Cygnus-X region comprising of a cluster of embedded protostars and young stellar objects located at a distance of 3.33 kpc. We investigate low-frequency radio emission from the protostellar jet associated with RAFGL2591 using the Giant Metrewave Radio Telescope (GMRT) at 325, 610 and 1280 MHz. For the first time, we have detected radio jet lobes in the E-W direction, labelled as GMRT-1 and GMRT-2. While GMRT-1 displays a flat radio spectral index of $\alpha$ = -0.10 , GMRT-2 shows a steeply negative value $\alpha$ = -0.62 suggestive of non-thermal emission. H$_2$ emission maps show the presence of numerous knots, arcs and extended emission towards the East-West jet, excited by the protostar VLA 3. In addition, we report a few H$_2$ knots in the North-East and South-West for the first time. The radio lobes (GMRT-1, GMRT-2) and H$_2$ emission towards this region are understood in the context of the prominent East-West jet as well as its lesser-known sibling jet in the North-East and South-West direction. To model the radio emission from the lobes, we have employed a numerical model including both thermal and non-thermal emission and found number densities towards these lobes in the range 100 - 1000 cm$^{-3}$ . The misalignment of the East-West jet lobes exhibits a reflection symmetry with a bending of $\sim$ 20$\circ$ . We attempt to understand this misalignment through precession caused by a binary partner and/or a supersonic side wind from source(s) in the vicinity., Comment: 15 pages, 9 figures, Accepted for publication in MNRAS

Cross-domain and cross-compositional generalization of Text-to-SQL semantic parsing is a challenging task. Existing Large Language Model (LLM) based solutions rely on inference-time retrieval of few-shot exemplars from the training set to synthesize a run-time prompt for each Natural Language (NL) test query. In contrast, we devise an algorithm which performs offline sampling of a minimal set-of few-shots from the training data, with complete coverage of SQL clauses, operators and functions, and maximal domain coverage within the allowed token length. This allows for synthesis of a fixed Generic Prompt (GP), with a diverse set-of exemplars common across NL test queries, avoiding expensive test time exemplar retrieval. We further auto-adapt the GP to the target database domain (DA-GP), to better handle cross-domain generalization; followed by a decomposed Least-To-Most-Prompting (LTMP-DA-GP) to handle cross-compositional generalization. The synthesis of LTMP-DA-GP is an offline task, to be performed one-time per new database with minimal human intervention. Our approach demonstrates superior performance on the KaggleDBQA dataset, designed to evaluate generalizability for the Text-to-SQL task. We further showcase consistent performance improvement of LTMP-DA-GP over GP, across LLMs and databases of KaggleDBQA, highlighting the efficacy and model agnostic benefits of our prompt based adapt and decompose approach., Comment: 22 Pages

Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks. However, it remains unclear whether models truly understand task definitions and whether the human-written definitions are optimal. In this paper, we systematically study the role of task definitions in instruction learning. We first conduct an ablation analysis informed by human annotations to understand which parts of a task definition are most important, and find that model performance only drops substantially when removing contents describing the task output, in particular label information. Next, we propose an automatic algorithm to compress task definitions to a minimal supporting set of tokens, and find that 60\% of tokens can be removed while maintaining or even improving model performance. Based on these results, we propose two strategies to help models better leverage task instructions: (1) providing only key information for tasks in a common structured format, and (2) adding a meta-tuning stage to help the model better understand the definitions. With these two strategies, we achieve a 4.2 Rouge-L improvement over 119 unseen test tasks., Comment: ACL 2023, camera-ready; 10 pages

Text simplification research has mostly focused on sentence-level simplification, even though many desirable edits - such as adding relevant background information or reordering content - may require document-level context. Prior work has also predominantly framed simplification as a single-step, input-to-output task, only implicitly modeling the fine-grained, span-level edits that elucidate the simplification process. To address both gaps, we introduce the SWiPE dataset, which reconstructs the document-level editing process from English Wikipedia (EW) articles to paired Simple Wikipedia (SEW) articles. In contrast to prior work, SWiPE leverages the entire revision history when pairing pages in order to better identify simplification edits. We work with Wikipedia editors to annotate 5,000 EW-SEW document pairs, labeling more than 40,000 edits with proposed 19 categories. To scale our efforts, we propose several models to automatically label edits, achieving an F-1 score of up to 70.6, indicating that this is a tractable but challenging NLU task. Finally, we categorize the edits produced by several simplification models and find that SWiPE-trained models generate more complex edits while reducing unwanted edits., Comment: ACL 2023, Long Paper

In this paper, we define a neuro-symbolic approach to address the task of finding semantically similar clones for the codes of the legacy programming language COBOL, without training data. We define a meta-model that is instantiated to have an Intermediate Representation (IR) in the form of Abstract Syntax Trees (ASTs) common across codes in C and COBOL. We linearize the IRs using Structure Based Traversal (SBT) to create sequential inputs. We further fine-tune UnixCoder, the best-performing model for zero-shot cross-programming language code search, for the Code Cloning task with the SBT IRs of C code-pairs, available in the CodeNet dataset. This allows us to learn latent representations for the IRs of the C codes, which are transferable to the IRs of the COBOL codes. With this fine-tuned UnixCoder, we get a performance improvement of 12.85 MAP@2 over the pre-trained UniXCoder model, in a zero-shot setting, on the COBOL test split synthesized from the CodeNet dataset. This demonstrates the efficacy of our meta-model based approach to facilitate cross-programming language transfer., Comment: 10 pages, 4 tables, 2 figures

Biological membranes are self-assembled complex fluid interfaces that host proteins, molecular motors and other macromolecules essential for cellular function. These membranes have a distinct in-plane fluid response with a surface viscosity that has been well characterized. The resulting quasi-2D fluid dynamical problem describes the motion of embedded proteins or particles. However, the viscous response of biological membranes is often non-Newtonian: in particular, the surface shear viscosity of phospholipids that comprise the membrane depends strongly on the surface pressure. We use the Lorentz reciprocal theorem to extract the effective long-ranged hydrodynamic interaction among membrane inclusions that arises due to such non-trivial rheology. We show that the corrective force that emerges ties back to the interplay between membrane flow and non-constant viscosity, which suggests a mechanism for biologically favorable protein aggregation within membranes. We quantify and describe the mechanism for such a large-scale concentration instability using a mean-field model. Finally, we employ numerical simulations to demonstrate the formation of hexatic crystals due to the effective hydrodynamic interactions within the membrane.

The spread of many infectious diseases is modeled using variants of the SIR compartmental model, which is a coupled differential equation. The coefficients of the SIR model determine the spread trajectories of disease, on whose basis proactive measures can be taken. Hence, the coefficient estimates must be both fast and accurate. Shaier et al. in the paper "Disease Informed Neural Networks" used Physics Informed Neural Networks (PINNs) to estimate the parameters of the SIR model. There are two drawbacks to this approach. First, the training time for PINNs is high, with certain diseases taking close to 90 hrs to train. Second, PINNs don't generalize for a new SIDR trajectory, and learning its corresponding SIR parameters requires retraining the PINN from scratch. In this work, we aim to eliminate both of these drawbacks. We generate a dataset between the parameters of ODE and the spread trajectories by solving the forward problem for a large distribution of parameters using the LSODA algorithm. We then use a neural network to learn the mapping between spread trajectories and coefficients of SIDR in an offline manner. This allows us to learn the parameters of a new spread trajectory without having to retrain, enabling generalization at test time. We observe a speed-up of 3-4 orders of magnitude with accuracy comparable to that of PINNs for 11 highly infectious diseases. Further finetuning of neural network inferred ODE coefficients using PINN further leads to 2-3 orders improvement of estimated coefficients., Comment: Publication accepted at International Conference for Learning Representations 2023: First Workshop in Machine Learning and Global Health

Protostellar jets are one of the primary signposts of star formation. A handful of protostellar objects exhibit radio emission from ionized jets, of which a few display negative spectral indices, indicating the presence of synchrotron emission. In this study, we characterize the radio spectra of HH80-81 jet with the help of a numerical model that we have developed earlier, which takes into account both thermal free-free and non-thermal synchrotron emission mechanisms. For modeling the HH80-81 jet, we consider jet emission towards the central region close to the driving source along with two Herbig-Haro objects, HH80 and HH81. We have obtained the best-fit parameters for each of these sources by fitting the model to radio observational data corresponding to two frequency windows taken across two epochs. Considering an electron number density in the range $10^3 - 10^5$ cm$^{-3}$, we obtained the thickness of the jet edges and fraction of relativistic electrons that contribute to non-thermal emission in the range $0.01^{\circ} - 0.1^{\circ}$ and $10^{-7} - 10^{-4}$, respectively. For the best-fit parameter sets, the model spectral indices lie in the range of -0.15 to +0.11 within the observed frequency windows., Comment: 14 pages, 6 figures, Accepted for publication in the Journal of Astrophysics and Astronomy

Generation of pseudo-code descriptions of legacy source code for software maintenance is a manually intensive task. Recent encoder-decoder language models have shown promise for automating pseudo-code generation for high resource programming languages such as C++, but are heavily reliant on the availability of a large code-pseudocode corpus. Soliciting such pseudocode annotations for codes written in legacy programming languages (PL) is a time consuming and costly affair requiring a thorough understanding of the source PL. In this paper, we focus on transferring the knowledge acquired by the code-to-pseudocode neural model trained on a high resource PL (C++) using parallel code-pseudocode data. We aim to transfer this knowledge to a legacy PL (C) with no PL-pseudocode parallel data for training. To achieve this, we utilize an Iterative Back Translation (IBT) approach with a novel test-cases based filtration strategy, to adapt the trained C++-to-pseudocode model to C-to-pseudocode model. We observe an improvement of 23.27% in the success rate of the generated C codes through back translation, over the successive IBT iteration, illustrating the efficacy of our approach., Comment: 11 pages, 1 figure, 5 tables

Physics-informed Neural Networks (PINNs) have been widely used to obtain accurate neural surrogates for a system of Partial Differential Equations (PDE). One of the major limitations of PINNs is that the neural solutions are challenging to interpret, and are often treated as black-box solvers. While Symbolic Regression (SR) has been studied extensively, very few works exist which generate analytical expressions to directly perform SR for a system of PDEs. In this work, we introduce an end-to-end framework for obtaining mathematical expressions for solutions of PDEs. We use a trained PINN to generate a dataset, upon which we perform SR. We use a Differentiable Program Architecture (DPA) defined using context-free grammar to describe the space of symbolic expressions. We improve the interpretability by pruning the DPA in a depth-first manner using the magnitude of weights as our heuristic. On average, we observe a 95.3% reduction in parameters of DPA while maintaining accuracy at par with PINNs. Furthermore, on an average, pruning improves the accuracy of DPA by 7.81% . We demonstrate our framework outperforms the existing state-of-the-art SR solvers on systems of complex PDEs like Navier-Stokes: Kovasznay flow and Taylor-Green Vortex flow. Furthermore, we produce analytical expressions for a complex industrial use-case of an Air-Preheater, without suffering from performance loss viz-a-viz PINNs., Comment: Publication accepted at International Conference for Learning Representations 2023: Physics for Machine Learning

Deep neural network (DNN) models for retinopathy have estimated predictive accuracies in the mid-to-high 90%. However, the following aspects remain unaddressed: State-of-the-art models are complex and require substantial computational infrastructure to train and deploy; The reliability of predictions can vary widely. In this paper, we focus on these aspects and propose a form of iterative knowledge distillation(IKD), called IKD+ that incorporates a tradeoff between size, accuracy and reliability. We investigate the functioning of IKD+ using two widely used techniques for estimating model calibration (Platt-scaling and temperature-scaling), using the best-performing model available, which is an ensemble of EfficientNets with approximately 100M parameters. We demonstrate that IKD+ equipped with temperature-scaling results in models that show up to approximately 500-fold decreases in the number of parameters than the original ensemble without a significant loss in accuracy. In addition, calibration scores (reliability) for the IKD+ models are as good as or better than the base mode, Comment: Submitted to IEEE International Conference on Image Processing (ICIP 2023)

Whole Slide Images (WSIs) or histopathology images are used in digital pathology. WSIs pose great challenges to deep learning models for clinical diagnosis, owing to their size and lack of pixel-level annotations. With the recent advancements in computational pathology, newer multiple-instance learning-based models have been proposed. Multiple-instance learning for WSIs necessitates creating patches and uses the encoding of these patches for diagnosis. These models use generic pre-trained models (ResNet-50 pre-trained on ImageNet) for patch encoding. The recently proposed KimiaNet, a DenseNet121 model pre-trained on TCGA slides, is a domain-specific pre-trained model. This paper shows the effect of domain-specific pre-training on WSI classification. To investigate the effect of domain-specific pre-training, we considered the current state-of-the-art multiple-instance learning models, 1) CLAM, an attention-based model, and 2) TransMIL, a self-attention-based model, and evaluated the models' confidence and predictive performance in detecting primary brain tumors - gliomas. Domain-specific pre-training improves the confidence of the models and also achieves a new state-of-the-art performance of WSI-based glioma subtype classification, showing a high clinical applicability in assisting glioma diagnosis. We will publicly share our code and experimental results at https://github.com/soham-chitnis10/WSI-domain-specific., Comment: Accepted in EMBC 2023