Search

Accurate survival prediction is essential for personalized cancer treatment. However, genomic data - often a more powerful predictor than pathology data - is costly and inaccessible. We present the cross-modal genomic feature translation and alignment network for enhanced survival prediction from histopathology images (PathoGen-X). It is a deep learning framework that leverages both genomic and imaging data during training, relying solely on imaging data at testing. PathoGen-X employs transformer-based networks to align and translate image features into the genomic feature space, enhancing weaker imaging signals with stronger genomic signals. Unlike other methods, PathoGen-X translates and aligns features without projecting them to a shared latent space and requires fewer paired samples. Evaluated on TCGA-BRCA, TCGA-LUAD, and TCGA-GBM datasets, PathoGen-X demonstrates strong survival prediction performance, emphasizing the potential of enriched imaging models for accessible cancer prognosis.

A wide range of non-equilibrium phenomena in nature involve non-reciprocal interactions. To understand the novel behaviors that can emerge in such systems, finding tractable models is essential. With this goal, we introduce a non-reciprocal generalization of the kinetic Ising model in one dimension and solve it exactly. Our solution uncovers novel properties driven by non-reciprocity, such as underdamped phases, critically damped phases where a system of size $N$ is described by an $N^{th}$-order exceptional point, and wave phenomena influenced by the parity of $N$. Additionally, we examine the low-energy behavior of these systems in various limits, demonstrating that non-reciprocity leads to unique scaling behavior at zero temperature., Comment: 8 pages, LaTeX, 3 figues. arXiv admin note: text overlap with arXiv:2410.23615

Non-reciprocal interactions are a generic feature of non-equilibrium systems. We define a non-reciprocal generalization of the kinetic Ising model in one spatial dimension. We solve the model exactly using two different approaches for infinite, semi-infinite and finite systems with either periodic or open boundary conditions. The exact solution allows us to explore a range of novel phenomena tied to non-reciprocity like non-reciprocity induced frustration and wave phenomena with interesting parity-dependence for finite systems of size $N$. We study dynamical questions like the approach to equilibrium with various boundary conditions. We find new regimes, separated by $N^{th}$-order exceptional points, which can be classified as overdamped, underdamped and critically damped phases. Despite these new regimes, long-time order is only present at zero temperature. Additionally, we explore the low-energy behavior of the system in various limits, including the ageing and spatio-temporal Porod regimes, demonstrating that non-reciprocity induces unique scaling behavior at zero temperature. Lastly, we present general results for systems where spins interact with no more than two spins, outlining the conditions under which long-time order may exist., Comment: 72 pages, LaTeX, 13 figures

Foundation models applied to bio-molecular space hold promise to accelerate drug discovery. Molecular representation is key to building such models. Previous works have typically focused on a single representation or view of the molecules. Here, we develop a multi-view foundation model approach, that integrates molecular views of graph, image and text. Single-view foundation models are each pre-trained on a dataset of up to 200M molecules and then aggregated into combined representations. Our multi-view model is validated on a diverse set of 18 tasks, encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. We show that the multi-view models perform robustly and are able to balance the strengths and weaknesses of specific views. We then apply this model to screen compounds against a large (>100 targets) set of G Protein-Coupled receptors (GPCRs). From this library of targets, we identify 33 that are related to Alzheimer's disease. On this subset, we employ our model to identify strong binders, which are validated through structure-based modeling and identification of key binding motifs., Comment: 34 pages including supplement. 9 figures, 4 tables

As large language models (LLMs) become integral to recruitment processes, concerns about AI-induced bias have intensified. This study examines biases in candidate interview reports generated by Claude 3.5 Sonnet, GPT-4o, Gemini 1.5, and Llama 3.1 405B, focusing on characteristics such as gender, race, and age. We evaluate the effectiveness of LLM-based anonymization in reducing these biases. Findings indicate that while anonymization reduces certain biases, particularly gender bias, the degree of effectiveness varies across models and bias types. Notably, Llama 3.1 405B exhibited the lowest overall bias. Moreover, our methodology of comparing anonymized and non-anonymized data reveals a novel approach to assessing inherent biases in LLMs beyond recruitment applications. This study underscores the importance of careful LLM selection and suggests best practices for minimizing bias in AI applications, promoting fairness and inclusivity., Comment: 13 pages, 18 figures. This paper presents a technical analysis of bias in large language models, focusing on bias detection and mitigation

Machine Learning models are often trained on proprietary and private data that cannot be shared, though the trained models themselves are distributed openly assuming that sharing model weights is privacy preserving, as training data is not expected to be inferred from the model weights. In this paper, we present Training-Like Data Reconstruction (TLDR), a network inversion-based approach to reconstruct training-like data from trained models. To begin with, we introduce a comprehensive network inversion technique that learns the input space corresponding to different classes in the classifier using a single conditioned generator. While inversion may typically return random and arbitrary input images for a given output label, we modify the inversion process to incentivize the generator to reconstruct training-like data by exploiting key properties of the classifier with respect to the training data along with some prior knowledge about the images. To validate our approach, we conduct empirical evaluations on multiple standard vision classification datasets, thereby highlighting the potential privacy risks involved in sharing machine learning models.

We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user's image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models. All videos from this paper are available at https://go.fb.me/MovieGenResearchVideos.

Missing frequency channels pose a problem for estimating $P(k_\perp,k_\parallel)$ the redshifted 21-cm power spectrum (PS) from radio-interferometric visibility data. This is particularly severe for the Murchison Widefield Array (MWA), which has a periodic pattern of missing channels that introduce spikes along $k_\parallel$. The Tracking Tapered Gridded Estimator (TTGE) overcomes this by first correlating the visibilities in the frequency domain to estimate the multi-frequency angular power spectrum (MAPS) $C_\ell(\Delta\nu)$ that has no missing frequency separation $\Delta\nu$. We perform a Fourier transform along $\Delta\nu$ to estimate $P(k_\perp,k_\parallel)$. Considering our earlier work, simulations demonstrate that the TTGE can estimate $P(k_\perp,k_\parallel)$ without any artifacts due to the missing channels. However, the spikes were still found to persist for the actual data, which is foreground-dominated. The current work presents a detailed investigation considering both simulations and actual data. We find that the spikes arise due to a combination of the missing channels and the strong spectral dependence of the foregrounds. Based on this, we propose and demonstrate a technique to mitigate the spikes. Applying this, we find the values of $P(k_\perp,k_\parallel)$ in the region $0.004 \leq k_\perp \leq 0.048\,{\rm Mpc^{-1}}$ and $k_\parallel > 0.35 \,{\rm Mpc^{-1}}$ to be consistent with zero within the expected statistical fluctuations. We obtain the $2\sigma$ upper limit of $\Delta_{\rm UL}^2(k)=(918.17)^2\,{\rm mK^2}$ at $k=0.404\,{\rm Mpc^{-1}}$ for the mean squared brightness temperature fluctuations of the $z=8.2$ epoch of reionization (EoR) 21-cm signal. This upper limit is from just $\sim 17$ minutes of observation for a single pointing direction. We expect tighter constraints when we combine all $162$ different pointing directions of the drift scan observation., Comment: 11 pages, 16 figures, comments are welcome

Understanding the evolution of galaxies cannot exclude the important role played by the central supermassive black hole and the circumgalactic medium (CGM). Simulations have strongly suggested the negative feedback of AGN Jet/wind/outflows on the ISM/CGM of a galaxy leading to the eventual decline of star formation. However, no "smoking gun" evidence exists so far where relics of feedback, observed in any band, are consistent with the time scale of a major decline in star formation, in any sample of galaxies. Relics of any AGN-driven outflows will be observed as a faint and fuzzy structure which may be difficult to characterise by automated algorithms but trained citizen scientists can possibly perform better through their intuitive vision with additional heterogeneous data available anywhere on the Internet. RAD@home, launched on 15th April 2013, is not only the first Indian Citizen Science Research (CSR) platform in astronomy but also the only CSR publishing discoveries using any Indian telescope. We briefly report 11 CSR discoveries collected over the last eleven years. While searching for such relics we have spotted cases of offset relic lobes from elliptical and spiral, episodic radio galaxies with overlapping lobes as the host galaxy is in motion, large diffuse spiral-shaped emission, cases of jet-galaxy interaction, kinks and burls on the jets, a collimated synchrotron thread etc. Such exotic sources push the boundaries of our understanding of classical Seyferts and radio galaxies with jets and the process of discovery prepares the next generation for science with the upgraded GMRT and Square Kilometre Array Observatory (SKAO)., Comment: 14 pages, 8 figures. Accepted for publication in the Springer-Nature conference proceedings for "ISRA 2023: The Relativistic Universe: From Classical to Quantum Proceedings of the International Symposium on Recent Developments in Relativistic Astrophysics". Comments and collaborations, most welcome! Please visit #RADatHomeIndia website at radathomeindia.org

In this study, we investigate the performance of few-shot classification models across different domains, specifically natural images and histopathological images. We first train several few-shot classification models on natural images and evaluate their performance on histopathological images. Subsequently, we train the same models on histopathological images and compare their performance. We incorporated four histopathology datasets and one natural images dataset and assessed performance across 5-way 1-shot, 5-way 5-shot, and 5-way 10-shot scenarios using a selection of state-of-the-art classification techniques. Our experimental results reveal insights into the transferability and generalization capabilities of few-shot classification models between diverse image domains. We analyze the strengths and limitations of these models in adapting to new domains and provide recommendations for optimizing their performance in cross-domain scenarios. This research contributes to advancing our understanding of few-shot learning in the context of image classification across diverse domains.

Federated Learning (FL) facilitates data privacy by enabling collaborative in-situ training across decentralized clients. Despite its inherent advantages, FL faces significant challenges of performance and convergence when dealing with data that is not independently and identically distributed (non-i.i.d.). While previous research has primarily addressed the issue of skewed label distribution across clients, this study focuses on the less explored challenge of multi-domain FL, where client data originates from distinct domains with varying feature distributions. We introduce a novel method designed to address these challenges FedStein: Enhancing Multi-Domain Federated Learning Through the James-Stein Estimator. FedStein uniquely shares only the James-Stein (JS) estimates of batch normalization (BN) statistics across clients, while maintaining local BN parameters. The non-BN layer parameters are exchanged via standard FL techniques. Extensive experiments conducted across three datasets and multiple models demonstrate that FedStein surpasses existing methods such as FedAvg and FedBN, with accuracy improvements exceeding 14% in certain domains leading to enhanced domain generalization. The code is available at https://github.com/sunnyinAI/FedStein, Comment: 12 pages, 2 figures. Accepted at International Workshop on Federated Foundation Models In Conjunction with NeurIPS 2024 (FL@FM-NeurIPS'24)

We developed a software pipeline for quality control (QC) of histopathology whole slide images (WSIs) that segments various regions, such as blurs of different levels, tissue regions, tissue folds, and pen marks. Given the necessity and increasing availability of GPUs for processing WSIs, the proposed pipeline comprises multiple lightweight deep learning models to strike a balance between accuracy and speed. The pipeline was evaluated in all TCGAs, which is the largest publicly available WSI dataset containing more than 11,000 histopathological images from 28 organs. It was compared to a previous work, which was not based on deep learning, and it showed consistent improvement in segmentation results across organs. To minimize annotation effort for tissue and blur segmentation, annotated images were automatically prepared by mosaicking patches (sub-images) from various WSIs whose labels were identified using a patch classification tool HistoROI. Due to the generality of our trained QC pipeline and its extensive testing the potential impact of this work is broad. It can be used for automated pre-processing any WSI cohort to enhance the accuracy and reliability of large-scale histopathology image analysis for both research and clinical use. We have made the trained models, training scripts, training data, and inference results publicly available at https://github.com/abhijeetptl5/wsisegqc, which should enable the research community to use the pipeline right out of the box or further customize it to new datasets and applications in the future., Comment: 14 pages, 8 figures

We propose two new evaluation metrics to assess realness of generated images based on normalizing flows: a simpler and efficient flow-based likelihood distance (FLD) and a more exact dual-flow based likelihood distance (D-FLD). Because normalizing flows can be used to compute the exact likelihood, the proposed metrics assess how closely generated images align with the distribution of real images from a given domain. This property gives the proposed metrics a few advantages over the widely used Fr\'echet inception distance (FID) and other recent metrics. Firstly, the proposed metrics need only a few hundred images to stabilize (converge in mean), as opposed to tens of thousands needed for FID, and at least a few thousand for the other metrics. This allows confident evaluation of even small sets of generated images, such as validation batches inside training loops. Secondly, the network used to compute the proposed metric has over an order of magnitude fewer parameters compared to Inception-V3 used to compute FID, making it computationally more efficient. For assessing the realness of generated images in new domains (e.g., x-ray images), ideally these networks should be retrained on real images to model their distinct distributions. Thus, our smaller network will be even more advantageous for new domains. Extensive experiments show that the proposed metrics have the desired monotonic relationships with the extent of image degradation of various kinds., Comment: 15 pages, 16 figures

Histopathology whole slide images (WSIs) are being widely used to develop deep learning-based diagnostic solutions, especially for precision oncology. Most of these diagnostic softwares are vulnerable to biases and impurities in the training and test data which can lead to inaccurate diagnoses. For instance, WSIs contain multiple types of tissue regions, at least some of which might not be relevant to the diagnosis. We introduce HistoROI, a robust yet lightweight deep learning-based classifier to segregate WSI into six broad tissue regions -- epithelium, stroma, lymphocytes, adipose, artifacts, and miscellaneous. HistoROI is trained using a novel human-in-the-loop and active learning paradigm that ensures variations in training data for labeling-efficient generalization. HistoROI consistently performs well across multiple organs, despite being trained on only a single dataset, demonstrating strong generalization. Further, we have examined the utility of HistoROI in improving the performance of downstream deep learning-based tasks using the CAMELYON breast cancer lymph node and TCGA lung cancer datasets. For the former dataset, the area under the receiver operating characteristic curve (AUC) for metastasis versus normal tissue of a neural network trained using weakly supervised learning increased from 0.88 to 0.92 by filtering the data using HistoROI. Similarly, the AUC increased from 0.88 to 0.93 for the classification between adenocarcinoma and squamous cell carcinoma on the lung cancer dataset. We also found that the performance of the HistoROI improves upon HistoQC for artifact detection on a test dataset of 93 annotated WSIs. The limitations of the proposed model are analyzed, and potential extensions are also discussed., Comment: 18 pages

Federated learning faces a critical challenge in balancing communication efficiency with rapid convergence, especially for second-order methods. While Newton-type algorithms achieve linear convergence in communication rounds, transmitting full Hessian matrices is often impractical due to quadratic complexity. We introduce Federated Learning with Enhanced Nesterov-Newton Sketch (FLeNS), a novel method that harnesses both the acceleration capabilities of Nesterov's method and the dimensionality reduction benefits of Hessian sketching. FLeNS approximates the centralized Newton's method without relying on the exact Hessian, significantly reducing communication overhead. By combining Nesterov's acceleration with adaptive Hessian sketching, FLeNS preserves crucial second-order information while preserving the rapid convergence characteristics. Our theoretical analysis, grounded in statistical learning, demonstrates that FLeNS achieves super-linear convergence rates in communication rounds - a notable advancement in federated optimization. We provide rigorous convergence guarantees and characterize tradeoffs between acceleration, sketch size, and convergence speed. Extensive empirical evaluation validates our theoretical findings, showcasing FLeNS's state-of-the-art performance with reduced communication requirements, particularly in privacy-sensitive and edge-computing scenarios. The code is available at https://github.com/sunnyinAI/FLeNS, Comment: 10 pages, 3 figures, 2 Tables

Current video summarization methods largely rely on transformer-based architectures, which, due to their quadratic complexity, require substantial computational resources. In this work, we address these inefficiencies by enhancing the Direct-to-Summarize Network (DSNet) with more resource-efficient token mixing mechanisms. We show that replacing traditional attention with alternatives like Fourier, Wavelet transforms, and Nystr\"omformer improves efficiency and performance. Furthermore, we explore various pooling strategies within the Regional Proposal Network, including ROI pooling, Fast Fourier Transform pooling, and flat pooling. Our experimental results on TVSum and SumMe datasets demonstrate that these modifications significantly reduce computational costs while maintaining competitive summarization performance. Thus, our work offers a more scalable solution for video summarization tasks., Comment: 10 pages, 5 figures

Recent advancements in single image super-resolution have been predominantly driven by token mixers and transformer architectures. WaveMixSR utilized the WaveMix architecture, employing a two-dimensional discrete wavelet transform for spatial token mixing, achieving superior performance in super-resolution tasks with remarkable resource efficiency. In this work, we present an enhanced version of the WaveMixSR architecture by (1) replacing the traditional transpose convolution layer with a pixel shuffle operation and (2) implementing a multistage design for higher resolution tasks ($4\times$). Our experiments demonstrate that our enhanced model -- WaveMixSR-V2 -- outperforms other architectures in multiple super-resolution tasks, achieving state-of-the-art for the BSD100 dataset, while also consuming fewer resources, exhibits higher parameter efficiency, lower latency and higher throughput. Our code is available at https://github.com/pranavphoenix/WaveMixSR., Comment: 10 pages. Accepted in AAAI 2025. arXiv admin note: text overlap with arXiv:2307.00430

Quantum error correction (QEC) is necessary to run large scale quantum programs. Regardless of error correcting code, hardware platform, or systems architecture, QEC systems are limited by the types of gates which they can perform efficiently. In order to make the base code's gate set universal, they typically rely on the production of a single type of resource state, commonly T, in a different code which is then distilled and injected into the base code. This process is neither space nor time efficient and can account for a large portion of the total execution time and physical qubit cost of any program. In order to circumvent this problem, alternatives have been proposed, such as the production of continuous angle rotation states \cite{akahoshi2023partially, choi2023fault}. These proposals are powerful because they not only enable localized resource generation but also can potentially reduce total space requirements. However, the production of these states is non-deterministic and can require many repetitions in order to obtain the desired resource. The original proposals suggest architectures which do not actively account for realtime management of its resources to minimize total execution time. Without this, static compilation of programs to these systems will be unnecessarily expensive. In this work, we propose a realtime compilation of programs to these continuous angle systems and a generalized resource sharing architecture which actively minimizes total execution time based on expected production rates. To do so, we repeatedly redistribute resources on-demand which depending on the underlying hardware can cause excessive classical control overhead. We further address this by dynamically selecting the frequency of recompilation. Our compiler and architecture improves over the baseline proposals by an average of $2\times$., Comment: 14 pages, 14 figures

The current standard for detecting human epidermal growth factor receptor 2 (HER2) status in breast cancer patients relies on HER2 amplification, identified through fluorescence in situ hybridization (FISH) or immunohistochemistry (IHC). However, hematoxylin and eosin (H\&E) tumor stains are more widely available, and accurately predicting HER2 status using H\&E could reduce costs and expedite treatment selection. Deep Learning algorithms for H&E have shown effectiveness in predicting various cancer features and clinical outcomes, including moderate success in HER2 status prediction. In this work, we employed a customized weak supervision classification technique combined with MoCo-v2 contrastive learning to predict HER2 status. We trained our pipeline on 182 publicly available H&E Whole Slide Images (WSIs) from The Cancer Genome Atlas (TCGA), for which annotations by the pathology team at Yale School of Medicine are publicly available. Our pipeline achieved an Area Under the Curve (AUC) of 0.85 across four different test folds. Additionally, we tested our model on 44 H&E slides from the TCGA-BRCA dataset, which had an HER2 score of 2+ and included corresponding HER2 status and FISH test results. These cases are considered equivocal for IHC, requiring an expensive FISH test on their IHC slides for disambiguation. Our pipeline demonstrated an AUC of 0.81 on these challenging H&E slides. Reducing the need for FISH test can have significant implications in cancer treatment equity for underserved populations.

This paper presents a study on few-shot classification in the context of histopathology images. While few-shot learning has been studied for natural image classification, its application to histopathology is relatively unexplored. Given the scarcity of labeled data in medical imaging and the inherent challenges posed by diverse tissue types and data preparation techniques, this research evaluates the performance of state-of-the-art few-shot learning methods for various scenarios on histology data. We have considered four histopathology datasets for few-shot histopathology image classification and have evaluated 5-way 1-shot, 5-way 5-shot and 5-way 10-shot scenarios with a set of state-of-the-art classification techniques. The best methods have surpassed an accuracy of 70%, 80% and 85% in the cases of 5-way 1-shot, 5-way 5-shot and 5-way 10-shot cases, respectively. We found that for histology images popular meta-learning approaches is at par with standard fine-tuning and regularization methods. Our experiments underscore the challenges of working with images from different domains and underscore the significance of unbiased and focused evaluations in advancing computer vision techniques for specialized domains, such as histology images.

Volunteer crowdsourcing (VCS) leverages citizen interaction to address challenges by utilizing individuals' knowledge and skills. Complex social tasks often require collaboration among volunteers with diverse skill sets, and their willingness to engage is crucial. Matching tasks with the most suitable volunteers remains a significant challenge. VCS platforms face unpredictable demands in terms of tasks and volunteer requests, complicating the prediction of resource requirements for the volunteer-to-task assignment process. To address these challenges, we introduce the Skill and Willingness-Aware Volunteer Matching (SWAM) algorithm, which allocates volunteers to tasks based on skills, willingness, and task requirements. We also developed a serverless framework to deploy SWAM. Our method outperforms conventional solutions, achieving a 71% improvement in end-to-end latency efficiency. We achieved a 92% task completion ratio and reduced task waiting time by 56%, with an overall utility gain 30% higher than state-of-the-art baseline methods. This framework contributes to generating effective volunteer and task matches, supporting grassroots community coordination and fostering citizen involvement, ultimately contributing to social good.

Previous methods for audio-image matching generally fall into one of two categories: pipeline models or End-to-End models. Pipeline models first transcribe speech and then encode the resulting text; End-to-End models encode speech directly. Generally, pipeline models outperform end-to-end models, but the intermediate transcription necessarily discards some potentially useful non-textual information. In addition to textual information, speech can convey details such as accent, mood, and and emphasis, which should be effectively captured in the encoded representation. In this paper, we investigate whether non-textual information, which is overlooked by pipeline-based models, can be leveraged to improve speech-image matching performance. We thoroughly analyze and compare End-to-End models, pipeline models, and our proposed dual-channel model for robust audio-image retrieval on a variety of datasets. Our approach achieves a substantial performance gain over the previous state-of-the-art by leveraging strong pretrained models, a prompting mechanism and a bifurcated design.

Classical optimization and learning-based methods are the two reigning paradigms in deformable image registration. While optimization-based methods boast generalizability across modalities and robust performance, learning-based methods promise peak performance, incorporating weak supervision and amortized optimization. However, the exact conditions for either paradigm to perform well over the other are shrouded and not explicitly outlined in the existing literature. In this paper, we make an explicit correspondence between the mutual information of the distribution of per-pixel intensity and labels, and the performance of classical registration methods. This strong correlation hints to the fact that architectural designs in learning-based methods is unlikely to affect this correlation, and therefore, the performance of learning-based methods. This hypothesis is thoroughly validated with state-of-the-art classical and learning-based methods. However, learning-based methods with weak supervision can perform high-fidelity intensity and label registration, which is not possible with classical methods. Next, we show that this high-fidelity feature learning does not translate to invariance to domain shift, and learning-based methods are sensitive to such changes in the data distribution. Finally, we propose a general recipe to choose the best paradigm for a given registration problem, based on these observations., Comment: 16 pages; Accepted to NeurIPS 2024

In-situ sensing devices need to be deployed in remote environments for long periods of time; minimizing their power consumption is vital for maximising both their operational lifetime and coverage. We introduce Terracorder -- a versatile multi-sensor device -- and showcase its exceptionally low power consumption using an on-device reinforcement learning scheduler. We prototype a unique device setup for biodiversity monitoring and compare its battery life using our scheduler against a number of fixed schedules; the scheduler captures more than 80% of events at less than 50% of the number of activations of the best-performing fixed schedule. We then explore how a collaborative scheduler can maximise the useful operation of a network of devices, improving overall network power consumption and robustness., Comment: Preprint

One of the few cases of AdS/CFT where both sides of the duality are under good control relates tensionless $k=1$ strings on AdS$_3$ to a two-dimensional symmetric product CFT. Building on prior observations, we propose an exact duality between string theory on a spacetime which is not asymptotically AdS and a non-conformal field theory. The bulk theory is constructed as a marginal deformation of the $k=1$ AdS$_3$ string while the spacetime dual is a single trace $T\overline{T}$-deformed symmetric orbifold theory. As evidence for the duality, we match the one-loop bulk and boundary torus partition functions. This correspondence provides a framework to both learn about quantum gravity beyond AdS and understand how to define physical observables in $T\overline{T}$-deformed field theories., Comment: 32 pages

Neural networks have emerged as powerful tools across various applications, yet their decision-making process often remains opaque, leading to them being perceived as "black boxes." This opacity raises concerns about their interpretability and reliability, especially in safety-critical scenarios. Network inversion techniques offer a solution by allowing us to peek inside these black boxes, revealing the features and patterns learned by the networks behind their decision-making processes and thereby provide valuable insights into how neural networks arrive at their conclusions, making them more interpretable and trustworthy. This paper presents a simple yet effective approach to network inversion using a carefully conditioned generator that learns the data distribution in the input space of the trained neural network, enabling the reconstruction of inputs that would most likely lead to the desired outputs. To capture the diversity in the input space for a given output, instead of simply revealing the conditioning labels to the generator, we hideously encode the conditioning label information into vectors, further exemplified by heavy dropout in the generation process and minimisation of cosine similarity between the features corresponding to the generated images. The paper concludes with immediate applications of Network Inversion including in interpretability, explainability and generation of adversarial samples.

Large Language Models (LLMs) have significantly transformed the educational landscape, offering new tools for students, instructors, and teaching assistants. This paper investigates the application of LLMs in assisting teaching assistants (TAs) with viva and code assessments in an advanced computing class on distributed systems in an Indian University. We develop TAMIGO, an LLM-based system for TAs to evaluate programming assignments. For viva assessment, the TAs generated questions using TAMIGO and circulated these questions to the students for answering. The TAs then used TAMIGO to generate feedback on student answers. For code assessment, the TAs selected specific code blocks from student code submissions and fed it to TAMIGO to generate feedback for these code blocks. The TAMIGO-generated feedback for student answers and code blocks was used by the TAs for further evaluation. We evaluate the quality of LLM-generated viva questions, model answers, feedback on viva answers, and feedback on student code submissions. Our results indicate that LLMs are highly effective at generating viva questions when provided with sufficient context and background information. However, the results for LLM-generated feedback on viva answers were mixed; instances of hallucination occasionally reduced the accuracy of feedback. Despite this, the feedback was consistent, constructive, comprehensive, balanced, and did not overwhelm the TAs. Similarly, for code submissions, the LLM-generated feedback was constructive, comprehensive and balanced, though there was room for improvement in aligning the feedback with the instructor-provided rubric for code evaluation. Our findings contribute to understanding the benefits and limitations of integrating LLMs into educational settings., Comment: Under review

Federated Learning (FL) offers a privacy-preserving approach to train models on decentralized data. Its potential in healthcare is significant, but challenges arise due to cross-client variations in medical image data, exacerbated by limited annotations. This paper introduces Cross-Client Variations Adaptive Federated Learning (CCVA-FL) to address these issues. CCVA-FL aims to minimize cross-client variations by transforming images into a common feature space. It involves expert annotation of a subset of images from each client, followed by the selection of a client with the least data complexity as the target. Synthetic medical images are then generated using Scalable Diffusion Models with Transformers (DiT) based on the target client's annotated images. These synthetic images, capturing diversity and representing the original data, are shared with other clients. Each client then translates its local images into the target image space using image-to-image translation. The translated images are subsequently used in a federated learning setting to develop a server model. Our results demonstrate that CCVA-FL outperforms Vanilla Federated Averaging by effectively addressing data distribution differences across clients without compromising privacy., Comment: I found critical errors in the manuscript affecting its validity. I need to correct these before resubmitting. Major changes to methodology and results are underway, significantly altering the content. I will resubmit the revised version

Recent advancements in radiance fields have opened new avenues for creating high-quality 3D assets and scenes. Style transfer can enhance these 3D assets with diverse artistic styles, transforming creative expression. However, existing techniques are often slow or unable to localize style transfer to specific objects. We introduce StyleSplat, a lightweight method for stylizing 3D objects in scenes represented by 3D Gaussians from reference style images. Our approach first learns a photorealistic representation of the scene using 3D Gaussian splatting while jointly segmenting individual 3D objects. We then use a nearest-neighbor feature matching loss to finetune the Gaussians of the selected objects, aligning their spherical harmonic coefficients with the style image to ensure consistency and visual appeal. StyleSplat allows for quick, customizable style transfer and localized stylization of multiple objects within a scene, each with a different style. We demonstrate its effectiveness across various 3D scenes and styles, showcasing enhanced control and customization in 3D creation., Comment: for code and results, see http://bernard0047.github.io/stylesplat

The first star-forming objects which formed at high redshifts during the cosmic dawn (CD) also emitted photons between Lyman-$\alpha$ and Lyman-limit frequencies. These photons are instrumental in coupling the spin temperature of the neutral hydrogen (HI) atoms with the kinetic temperature of the intergalactic medium (IGM). Along with this coupling effect, these photons also impact the kinetic temperature by exchanging energy with the HI atoms. The injected Lyman-$\alpha$ photons in general cool the medium, while the continuum photons heat the medium. While studying this effect in the literature, quasi-static profile around the Lyman-$\alpha$ frequency is assumed. In this paper, we solve the time-dependent coupled dynamics of the photon intensity profile along with the evolution of the thermal state of the IGM and HI spin temperature. It is expected that, during the CD era, the IGM has a mix of continuum photons with 10-20% of injected photons. For this case, we show that the system reaches thermal equilibrium in around 1 Myr, with final temperature in the range 50-100 K. This time scale is comparable to the source lifetime of PopIII stars at high redshifts. One impact of switching off short-lived sources is that it can keep the system heated above the temperature of the quasi-static state. We also show that the quasi-static equilibrium for the continuum photons is only achieved on time scales of 100 Myr at $z\simeq 20$, comparable to the age of the Universe. We also briefly discuss how the Lyman-$\alpha$ induced heating can impact the 21 cm signal from CD., Comment: 18 pages, 6 figures, submitted to ApJ

In contemporary computer vision applications, particularly image classification, architectural backbones pre-trained on large datasets like ImageNet are commonly employed as feature extractors. Despite the widespread use of these pre-trained convolutional neural networks (CNNs), there remains a gap in understanding the performance of various resource-efficient backbones across diverse domains and dataset sizes. Our study systematically evaluates multiple lightweight, pre-trained CNN backbones under consistent training settings across a variety of datasets, including natural images, medical images, galaxy images, and remote sensing images. This comprehensive analysis aims to aid machine learning practitioners in selecting the most suitable backbone for their specific problem, especially in scenarios involving small datasets where fine-tuning a pre-trained network is crucial. Even though attention-based architectures are gaining popularity, we observed that they tend to perform poorly under low data finetuning tasks compared to CNNs. We also observed that some CNN architectures such as ConvNeXt, RegNet and EfficientNet performs well compared to others on a diverse set of domains consistently. Our findings provide actionable insights into the performance trade-offs and effectiveness of different backbones, facilitating informed decision-making in model selection for a broad spectrum of computer vision domains. Our code is available here: https://github.com/pranavphoenix/Backbones, Comment: 12 pages, 2 figures

Entropy regularization has been extensively used in policy optimization algorithms to regularize the optimization landscape and accelerate convergence; however, it comes at the cost of introducing an additional regularization bias. This work quantifies the impact of entropy regularization on the convergence of policy gradient methods for stochastic exit time control problems. We analyze a continuous-time policy mirror descent dynamics, which updates the policy based on the gradient of an entropy-regularized value function and adjusts the strength of entropy regularization as the algorithm progresses. We prove that with a fixed entropy level, the dynamics converges exponentially to the optimal solution of the regularized problem. We further show that when the entropy level decays at suitable polynomial rates, the annealed flow converges to the solution of the unregularized problem at a rate of $\mathcal O(1/S)$ for discrete action spaces and, under suitable conditions, at a rate of $\mathcal O(1/\sqrt{S})$ for general action spaces, with $S$ being the gradient flow time. This paper explains how entropy regularization improves policy optimization, even with the true gradient, from the perspective of convergence rate.

We report the discovery of an S-shaped morphology of the radio galaxy J0644$+$1043 imaged with a 30 $\mu$Jy sensitive 525 MHz broadband (band 3 $+$ 4) uGMRT map. Dedicated spectroscopic observations of the host galaxy carried out with the 2-meter Rozhen telescope yielded a redshift of 0.0488, giving a projected linear size of the peculiar radio structure of over 0.7 Mpc. This giant radio galaxy is powered by a black hole of mass 4.1$^{+9.39}_{-2.87}\times 10^8$ \msun, from which vicinity emanate well-collimated and knotty jets, each $\sim$100 kpc long. The entire radio structure, presumably due to the effective jet precession, is less than 50 Myr old, has a power of $\sim$6 $\times 10^{24}$ W Hz$^{-1}$ at 1.4 GHz and the observed morphological characteristics do not strictly conform to the traditional FR I or FR II categories.

In this work, we give generalization bounds of statistical learning algorithms trained on samples drawn from a dependent data source, both in expectation and with high probability, using the Online-to-Batch conversion paradigm. We show that the generalization error of statistical learners in the dependent data setting is equivalent to the generalization error of statistical learners in the i.i.d. setting up to a term that depends on the decay rate of the underlying mixing stochastic process and is independent of the complexity of the statistical learner. Our proof techniques involve defining a new notion of stability of online learning algorithms based on Wasserstein distances and employing "near-martingale" concentration bounds for dependent random variables to arrive at appropriate upper bounds for the generalization error of statistical learners trained on dependent data.

Drift scan observations provide the broad sky coverage and instrumental stability needed to measure the Epoch of Reionization (EoR) 21-cm signal. In such observations, the telescope's pointing center (PC) moves continuously on the sky. The Tracking Tapered Gridded Estimator (TTGE) combines observations from different PC to estimate $P(k_{\perp}, k_{\parallel})$ the 21-cm power spectrum, centered on a tracking center (TC) which remains fixed on the sky. The tapering further restricts the sky response to a small angular region around TC, thereby mitigating wide-field foregrounds. Here we consider $154.2 \, {\rm MHz}$ ($z = 8.2$) Murchison Widefield Array (MWA) drift scan observations. The periodic pattern of flagged channels, present in MWA data, is known to introduce artefacts which pose a challenge for estimating $P(k_{\perp}, k_{\parallel})$. We demonstrate that the TTGE is able to recover $P(k_{\perp}, k_{\parallel})$ without any artefacts, and estimate $P(k)$ within $5 \%$ accuracy over a large $k$-range. We also present preliminary results for a single PC, combining 9 nights of observation $(17 \, {\rm min}$ total). We find that $P(k_{\perp}, k_{\parallel})$ exhibits streaks at a fixed interval of $k_{\parallel}=0.29 \, {\rm Mpc}^{-1}$, which matches $\Delta \nu_{\rm per}=1.28 \, {\rm MHz}$ that is the period of the flagged channels. The streaks are not as pronounced at larger $k_{\parallel}$, and in some cases they do not appear to extend across the entire $k_{\perp}$ range. The rectangular region $0.05 \leq k_{\perp} \leq 0.16 \, {\rm Mpc^{-1}}$ and $0.9 \leq k_{\parallel} \leq 4.6 \, {\rm Mpc^{-1}}$ is found to be relatively free of foreground contamination and artefacts, and we have used this to place the $2\sigma$ upper limit $\Delta^2(k) < (1.85 \times 10^4)^2\, {\rm mK^2}$ on the EoR 21-cm mean squared brightness temperature fluctuations at $k=1 \,{\rm Mpc}^{-1}$., Comment: 15 pages, 11 figures, accepted for publication in PASA

Light mass warm dark matter is an interesting and viable alternative to the cold dark matter paradigm. An intriguing variation of this scenario is the mass-varying dark matter model where the dark matter mass varies with time during its cosmic history. This is realized in multiple particle physics models. In this work, we study the cosmological constraints on such a model where the dark matter mass transitions from zero to a finite value in the early Universe. In this model, the matter power spectrum exhibits power suppression below a scale that depends on the epoch of transition, and the angular power spectrum of the cosmic microwave background show a distinctive phase shift. We use the latest cosmic microwave background and the weak lensing data to place lower limit on the transition redshift and ease the $S_8$ tension, unlike the warm dark matter model. This analysis also facilitates a marginal detection of the dark matter (DM) mass. Our findings reveal that while Planck data alone reduces the $S_8$ tension to approximately $2\sigma$, it does not sufficiently constrain the DM mass. However, when combined with the $S_8$ measurement from KIDS1000+BOSS+2dfLenS, the tension significantly decreases to roughly $1.3\sigma$, and we observe the detection of a DM mass at $41.7^{+7.81}_{-27.5}\,\mathrm{eV}$. Further analysis incorporating a combined data set from ACT and weak lensing results in an even more pronounced reduction in the tension to approximately $0.4\sigma$, alongside a higher detected mass of $51.2^{+16}_{-33.5}\,\mathrm{eV}$. We also find a better fit to the combined data compared to the $\Lambda$CDM model., Comment: 10 pages, 6 figures, Reference updated

Large Language Models (LLMs) trained on code are revolutionizing the software development process. Increasingly, code LLMs are being integrated into software development environments to improve the productivity of human programmers, and LLM-based agents are beginning to show promise for handling complex tasks autonomously. Realizing the full potential of code LLMs requires a wide range of capabilities, including code generation, fixing bugs, explaining and documenting code, maintaining repositories, and more. In this work, we introduce the Granite series of decoder-only code models for code generative tasks, trained with code written in 116 programming languages. The Granite Code models family consists of models ranging in size from 3 to 34 billion parameters, suitable for applications ranging from complex application modernization tasks to on-device memory-constrained use cases. Evaluation on a comprehensive set of tasks demonstrates that Granite Code models consistently reaches state-of-the-art performance among available open-source code LLMs. The Granite Code model family was optimized for enterprise software development workflows and performs well across a range of coding tasks (e.g. code generation, fixing and explanation), making it a versatile all around code model. We release all our Granite Code models under an Apache 2.0 license for both research and commercial use., Comment: Corresponding Authors: Rameswar Panda, Ruchir Puri; Equal Contributors: Mayank Mishra, Matt Stallone, Gaoyuan Zhang

Video-based Question Answering (Video QA) is a challenging task and becomes even more intricate when addressing Socially Intelligent Question Answering (SIQA). SIQA requires context understanding, temporal reasoning, and the integration of multimodal information, but in addition, it requires processing nuanced human behavior. Furthermore, the complexities involved are exacerbated by the dominance of the primary modality (text) over the others. Thus, there is a need to help the task's secondary modalities to work in tandem with the primary modality. In this work, we introduce a cross-modal alignment and subsequent representation fusion approach that achieves state-of-the-art results (82.06\% accuracy) on the Social IQ 2.0 dataset for SIQA. Our approach exhibits an improved ability to leverage the video modality by using the audio modality as a bridge with the language modality. This leads to enhanced performance by reducing the prevalent issue of language overfitting and resultant video modality bypassing encountered by current existing techniques. Our code and models are publicly available at https://github.com/sts-vlcc/sts-vlcc

In recent years, the detection of gravitational waves by LIGO and PTA collaborations have raised the intriguing possibility of excess matter power at small scales. Such an increase can be achieved by ultra slow roll (USR) phase during inflationary epoch. We constrain excess power over small scales within the framework of such models using cosmological datasets, particularly of CMB anisotropies and Lyman-$\alpha$. We parameterize the USR phase in terms of the e-fold at the onset of USR (counted from the end of inflation) $\bar N_1$ and the duration of USR phase $\Delta N$. The former dictates the scale of enhancement in the primordial power spectrum, while the latter determines the amplitude of such an enhancement. From a joint dataset of CMB, SNIa and galaxy surveys, we obtain $\bar N_1 \lesssim 45$ with no bound on $\Delta N$. This in turn implies that the scales over which the power spectrum can deviate significantly from the nearly scale invariant behavior of a typical slow-roll model is $k \gtrsim 1 \, \rm Mpc^{-1}$. On the other hand, the Lyman-$\alpha$ data is sensitive to baryonic power spectrum along the line of sight. We consider a semi-analytic theoretical method and high spectral-resolution Lyman-$\alpha$ data to constrain the model. The Lyman-$\alpha$ data limits both the USR parameters: $\bar N_1 \lesssim 41$ and $\Delta N \lesssim 0.4$. This constrains the amplitude of the power spectrum enhancement to be less than a factor of hundred over scales $1 \lesssim k/{\rm Mpc^{-1}} \lesssim 100$, thereby considerably improving the constraint on power over these scales as compared to the bounds arrived at from CMB spectral distortion., Comment: v1: 27 pages, 8 figures; v2: 24 pages, 7 figures, updated dataset, discussion and references, accepted in JCAP

Training a computer vision system to segment a novel class typically requires collecting and painstakingly annotating lots of images with objects from that class. Few-shot segmentation techniques reduce the required number of images to learn to segment a new class, but careful annotations of object boundaries are still required. On the other hand, interactive segmentation techniques only focus on incrementally improving the segmentation of one object at a time (typically, using clicks given by an expert) in a class-agnostic manner. We combine the two concepts to drastically reduce the effort required to train segmentation models for novel classes. Instead of trivially feeding interactive segmentation masks as ground truth to a few-shot segmentation model, we propose IFSENet, which can accept sparse supervision on a single or few support images in the form of clicks to generate masks on support (training, at least clicked upon once) as well as query (test, never clicked upon) images. To trade-off effort for accuracy flexibly, the number of images and clicks can be incrementally added to the support set to further improve the segmentation of support as well as query images. The proposed model approaches the accuracy of previous state-of-the-art few-shot segmentation models with considerably lower annotation effort (clicks instead of maps), when tested on Pascal and SBD datasets on query images. It also works well as an interactive segmentation method on support images.

In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.

Stellar binaries are ubiquitous in the galaxy and a laboratory for astrophysical effects. We use TESS to study photometric modulations in the lightcurves of 162 unequal mass eclipsing binaries from the EBLM (Eclipsing Binary Low Mass) survey, comprising F/G/K primaries and M-dwarf secondaries. We detect modulations on 81 eclipsing binaries. We catalog the rotation rates of the primary star in 69 binaries and discover 17 ellipsoidal variables. In a large portion (at least $\sim 51\%$) of our sample, we detect photometric modulations consistent with two over-densities of spots on the primary star that are roughly $180^{\circ}$ apart. We show that these so-called active longitudes are preferentially at the sub- and anti-stellar points on the primary star. Physically, this means that the spots on the primary star preferentially face directly towards and away from the secondary star., Comment: 14 pages, 11 figures

Gene selection plays a pivotal role in oncology research for improving outcome prediction accuracy and facilitating cost-effective genomic profiling for cancer patients. This paper introduces two gene selection strategies for deep learning-based survival prediction models. The first strategy uses a sparsity-inducing method while the second one uses importance based gene selection for identifying relevant genes. Our overall approach leverages the power of deep learning to model complex biological data structures, while sparsity-inducing methods ensure the selection process focuses on the most informative genes, minimizing noise and redundancy. Through comprehensive experimentation on diverse genomic and survival datasets, we demonstrate that our strategy not only identifies gene signatures with high predictive power for survival outcomes but can also streamlines the process for low-cost genomic profiling. The implications of this research are profound as it offers a scalable and effective tool for advancing personalized medicine and targeted cancer therapies. By pushing the boundaries of gene selection methodologies, our work contributes significantly to the ongoing efforts in cancer genomics, promising improved diagnostic and prognostic capabilities in clinical settings.

We present INDICVOICES, a dataset of natural and spontaneous speech containing a total of 7348 hours of read (9%), extempore (74%) and conversational (17%) audio from 16237 speakers covering 145 Indian districts and 22 languages. Of these 7348 hours, 1639 hours have already been transcribed, with a median of 73 hours per language. Through this paper, we share our journey of capturing the cultural, linguistic and demographic diversity of India to create a one-of-its-kind inclusive and representative dataset. More specifically, we share an open-source blueprint for data collection at scale comprising of standardised protocols, centralised tools, a repository of engaging questions, prompts and conversation scenarios spanning multiple domains and topics of interest, quality control mechanisms, comprehensive transcription guidelines and transcription tools. We hope that this open source blueprint will serve as a comprehensive starter kit for data collection efforts in other multilingual regions of the world. Using INDICVOICES, we build IndicASR, the first ASR model to support all the 22 languages listed in the 8th schedule of the Constitution of India. All the data, tools, guidelines, models and other materials developed as a part of this work will be made publicly available

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.