Search

Large Language Models (LLMs) are increasingly integrated into diverse applications. The rapid evolution of LLMs presents opportunities for developers to enhance applications continuously. However, this constant adaptation can also lead to performance regressions during model migrations. While several interactive tools have been proposed to streamline the complexity of prompt engineering, few address the specific requirements of regression testing for LLM Migrations. To bridge this gap, we introduce RETAIN (REgression Testing guided LLM migrAtIoN), a tool designed explicitly for regression testing in LLM Migrations. RETAIN comprises two key components: an interactive interface tailored to regression testing needs during LLM migrations, and an error discovery module that facilitates understanding of differences in model behaviors. The error discovery module generates textual descriptions of various errors or differences between model outputs, providing actionable insights for prompt refinement. Our automatic evaluation and empirical user studies demonstrate that RETAIN, when compared to manual evaluation, enabled participants to identify twice as many errors, facilitated experimentation with 75% more prompts, and achieves 12% higher metric scores in a given time frame., Comment: Preprint

Modern web services adopt cloud-native principles to leverage the advantages of microservices. To consistently guarantee high Quality of Service (QoS) according to Service Level Agreements (SLAs), ensure satisfactory user experiences, and minimize operational costs, each microservice must be provisioned with the right amount of resources. However, accurately provisioning microservices with adequate resources is complex and depends on many factors, including workload intensity and the complex interconnections between microservices. To address this challenge, we develop a model that captures the relationship between an end-to-end latency, requests at the front-end level, and resource utilization. We then use the developed model to predict the end-to-end latency. Our solution leverages the Temporal Fusion Transformer (TFT), an attention-based architecture equipped with interpretability features. When the prediction results indicate SLA non-compliance, we use the feature importance provided by the TFT as covariates in Kernel Ridge Regression (KRR), with the response variable being the desired latency, to learn the parameters associated with the feature importance. These learned parameters reflect the adjustments required to the features to ensure SLA compliance. We demonstrate the merit of our approach with a microservice-based application and provide a roadmap to deployment., Comment: 14 pages, 5 figures

The magnetic cloud (MC) of the Coronal Mass Ejection on April 24, 2023, contains sub-Alfv\'{e}nic solar wind, transforming Earth's magnetosphere from conventional bow-shock magnetotail configuration to Alfv\'{e}n wings. Utilizing measurements from the Magnetosphere Multiscale (MMS) mission, we present for the first time electron distribution signatures as the spacecraft traverses through various magnetic topologies during this transformation. Specifically, we characterize electrons inside the sub-Alfv\'{e}nic MC, on the dawn-dusk wing field lines and on the closed field lines. The signatures include strahl electrons in MC regions and energetic keV electrons streaming along the dawn and dusk wing field lines. We demonstrate the distribution signatures of dual wing reconnection, defined as reconnection between dawn-dusk Alfv\'{e}n wing field lines and the IMF. These signatures include four electron populations comprised of partially-depleted MC electrons and bi-directional energetic electrons with variations in energy and pitch-angle. The distributions reveal evidence of bursty magnetic reconnection under northward IMF., Comment: 11pages 4 figures

As user content and queries become increasingly multi-modal, the need for effective multi-modal search systems has grown. Traditional search systems often rely on textual and metadata annotations for indexed images, while multi-modal embeddings like CLIP enable direct search using text and image embeddings. However, embedding-based approaches face challenges in integrating contextual features such as user locale and recency. Building a scalable multi-modal search system requires fine-tuning several components. This paper presents a multi-modal search architecture and a series of AB tests that optimize embeddings and multi-modal technologies in Adobe Express template search. We address considerations such as embedding model selection, the roles of embeddings in matching and ranking, and the balance between dense and sparse embeddings. Our iterative approach demonstrates how utilizing sparse, dense, and contextual features enhances short and long query search, significantly reduces null rates (over 70\%), and increases click-through rates (CTR). Our findings provide insights into developing robust multi-modal search systems, thereby enhancing relevance for complex queries., Comment: CIKM 2024 (International Conference on Information and Knowledge Management), Multimodal Search and Recommendations Workshop

Recent research has shown that smaller language models can acquire substantial reasoning abilities when fine-tuned with reasoning exemplars crafted by a significantly larger teacher model. We explore this paradigm for the financial domain, focusing on the challenge of answering questions that require multi-hop numerical reasoning over financial texts. We assess the performance of several smaller models that have been fine-tuned to generate programs that encode the required financial reasoning and calculations. Our findings demonstrate that these fine-tuned smaller models approach the performance of the teacher model. To provide a granular analysis of model performance, we propose an approach to investigate the specific student model capabilities that are enhanced by fine-tuning. Our empirical analysis indicates that fine-tuning refines the student models ability to express and apply the required financial concepts along with adapting the entity extraction for the specific data format. In addition, we hypothesize and demonstrate that comparable financial reasoning capability can be induced using relatively smaller datasets.

Planning remains one of the last standing bastions for large language models (LLMs), which now turn their attention to search. Most of the literature uses the language models as world models to define the search space, forgoing soundness for the sake of flexibility. A recent work, Thought of Search (ToS), proposed defining the search space with code, having the language models produce that code. ToS requires a human in the loop, collaboratively producing a sound successor function and goal test. The result, however, is worth the effort: all the tested datasets were solved with 100% accuracy. At the same time LLMs have demonstrated significant progress in code generation and refinement for complex reasoning tasks. In this work, we automate ToS (AutoToS), completely taking the human out of the loop of solving planning problems. AutoToS guides the language model step by step towards the generation of sound and complete search components, through feedback from both generic and domain specific unit tests. We achieve 100% accuracy, with minimal feedback iterations, using LLMs of various sizes on all evaluated domains.

Recent advancements in retrieval-augmented generation (RAG) have demonstrated impressive performance in the question-answering (QA) task. However, most previous works predominantly focus on text-based answers. While some studies address multimodal data, they still fall short in generating comprehensive multimodal answers, particularly for explaining concepts or providing step-by-step tutorials on how to accomplish specific goals. This capability is especially valuable for applications such as enterprise chatbots and settings such as customer service and educational systems, where the answers are sourced from multimodal data. In this paper, we introduce a simple and effective framework named MuRAR (Multimodal Retrieval and Answer Refinement). MuRAR enhances text-based answers by retrieving relevant multimodal data and refining the responses to create coherent multimodal answers. This framework can be easily extended to support multimodal answers in enterprise chatbots with minimal modifications. Human evaluation results indicate that multimodal answers generated by MuRAR are more useful and readable compared to plain text answers., Comment: Preprint

Recently, PdTe has been identified as a Dirac semimetal with potential for unconventional superconductivity based on ARPES measurements. This study presents electrical transport and magnetization measurements conducted on high-quality single crystals of PdTe. Anisotropy in the upper critical magnetic field is observed in resistivity versus temperature data measured under various applied magnetic fields for in-plane (B II ab ) and out-of-plane (B II c) orientations. The magnetic field versus temperature (H - T) phase diagram extracted from resistivity data exhibits an upward curvature akin to several multigap superconductors. Additionally, magnetization measurements reveal de Haas-Van Alphen (dHvA) oscillations in both B II ab and B II c orientations. Fourier analysis of the quantum oscillations identifies two Fermi pockets. Moreover, the Landau fan diagram for a small Fermi pocket confirms a non-trivial Berry phase {\pi}, indicative of the Dirac nature of PdTe. Based on quantum oscillation data, a plausible band diagram is constructed.

Time series forecasting is an important task in many fields ranging from supply chain management to weather forecasting. Recently, Transformer neural network architectures have shown promising results in forecasting on common time series benchmark datasets. However, application to supply chain demand forecasting, which can have challenging characteristics such as sparsity and cross-series effects, has been limited. In this work, we explore the application of Transformer-based models to supply chain demand forecasting. In particular, we develop a new Transformer-based forecasting approach using a shared, multi-task per-time series network with an initial component applying attention across time series, to capture interactions and help address sparsity. We provide a case study applying our approach to successfully improve demand prediction for a medical device manufacturing company. To further validate our approach, we also apply it to public demand forecasting datasets as well and demonstrate competitive to superior performance compared to a variety of baseline and state-of-the-art forecast methods across the private and public datasets.

Collaborative planning for congenital heart diseases typically involves creating physical heart models through 3D printing, which are then examined by both surgeons and cardiologists. Recent developments in mobile augmented reality (AR) technologies have presented a viable alternative, known for their ease of use and portability. However, there is still a lack of research examining the utilization of multi-user mobile AR environments to support collaborative planning for cardiovascular surgeries. We created ARCollab, an iOS AR app designed for enabling multiple surgeons and cardiologists to interact with a patient's 3D heart model in a shared environment. ARCollab enables surgeons and cardiologists to import heart models, manipulate them through gestures and collaborate with other users, eliminating the need for fabricating physical heart models. Our evaluation of ARCollab's usability and usefulness in enhancing collaboration, conducted with three cardiothoracic surgeons and two cardiologists, marks the first human evaluation of a multi-user mobile AR tool for surgical planning. ARCollab is open-source, available at https://github.com/poloclub/arcollab.

The rapid growth of Artificial Intelligence (AI) has underscored the urgent need for responsible AI practices. Despite increasing interest, a comprehensive AI risk assessment toolkit remains lacking. This study introduces our Responsible AI (RAI) Question Bank, a comprehensive framework and tool designed to support diverse AI initiatives. By integrating AI ethics principles such as fairness, transparency, and accountability into a structured question format, the RAI Question Bank aids in identifying potential risks, aligning with emerging regulations like the EU AI Act, and enhancing overall AI governance. A key benefit of the RAI Question Bank is its systematic approach to linking lower-level risk questions to higher-level ones and related themes, preventing siloed assessments and ensuring a cohesive evaluation process. Case studies illustrate the practical application of the RAI Question Bank in assessing AI projects, from evaluating risk factors to informing decision-making processes. The study also demonstrates how the RAI Question Bank can be used to ensure compliance with standards, mitigate risks, and promote the development of trustworthy AI systems. This work advances RAI by providing organizations with a valuable tool to navigate the complexities of ethical AI development and deployment while ensuring comprehensive risk management., Comment: 30 pages, 6 tables, 14 figures

Artificial Intelligence (AI) is a widely developed and adopted technology across entire industry sectors. Integrating environmental, social, and governance (ESG) considerations with AI investments is crucial for ensuring ethical and sustainable technological advancement. Particularly from an investor perspective, this integration not only mitigates risks but also enhances long-term value creation by aligning AI initiatives with broader societal goals. Yet, this area has been less explored in both academia and industry. To bridge the gap, we introduce a novel ESG-AI framework, which is developed based on insights from engagements with 28 companies and comprises three key components. The framework provides a structured approach to this integration, developed in collaboration with industry practitioners. The ESG-AI framework provides an overview of the environmental and social impacts of AI applications, helping users such as investors assess the materiality of AI use. Moreover, it enables investors to evaluate a company's commitment to responsible AI through structured engagements and thorough assessment of specific risk areas. We have publicly released the framework and toolkit in April 2024, which has received significant attention and positive feedback from the investment community. This paper details each component of the framework, demonstrating its applicability in real-world contexts and its potential to guide ethical AI investments., Comment: 23 pages, 8 tables, 10 figures

This paper presents a comprehensive survey of AI-driven mini-grid solutions aimed at enhancing sustainable energy access. It emphasises the potential of mini-grids, which can operate independently or in conjunction with national power grids, to provide reliable and affordable electricity to remote communities. Given the inherent unpredictability of renewable energy sources such as solar and wind, the necessity for accurate energy forecasting and management is discussed, highlighting the role of advanced AI techniques in forecasting energy supply and demand, optimising grid operations, and ensuring sustainable energy distribution. This paper reviews various forecasting models, including statistical methods, machine learning algorithms, and hybrid approaches, evaluating their effectiveness for both short-term and long-term predictions. Additionally, it explores public datasets and tools such as Prophet, NeuralProphet, and N-BEATS for model implementation and validation. The survey concludes with recommendations for future research, addressing challenges in model adaptation and optimisation for real-world applications.

Mechanistic interpretability has been explored in detail for large language models (LLMs). For the first time, we provide a preliminary investigation with similar interpretability methods for medical imaging. Specifically, we analyze the features from a ViT-Small encoder obtained from a pathology Foundation Model via application to two datasets: one dataset of pathology images, and one dataset of pathology images paired with spatial transcriptomics. We discover an interpretable representation of cell and tissue morphology, along with gene expression within the model embedding space. Our work paves the way for further exploration around interpretable feature dimensions and their utility for medical and clinical applications.

We explore the recently proposed explainable acoustic neural embedding~(XANE) system that models the background acoustics of a speech signal in a non-intrusive manner. The XANE embeddings are used to estimate specific parameters related to the background acoustic properties of the signal which allows the embeddings to be explainable in terms of those parameters. We perform ablation studies on the XANE system and show that estimating all acoustic parameters jointly has an overall positive effect. Furthermore, we illustrate the value of XANE embeddings by performing clustering experiments on unseen test data and show that the proposed embeddings achieve a mean F1 score of 92\% for three different tasks, outperforming significantly the WavLM based signal embeddings and are complimentary to speaker embeddings., Comment: arXiv admin note: substantial text overlap with arXiv:2406.05199

Enterprises have a growing need to identify relevant tables in data lakes; e.g. tables that are unionable, joinable, or subsets of each other. Tabular neural models can be helpful for such data discovery tasks. In this paper, we present TabSketchFM, a neural tabular model for data discovery over data lakes. First, we propose novel pre-training: a sketch-based approach to enhance the effectiveness of data discovery in neural tabular models. Second, we finetune the pretrained model for identifying unionable, joinable, and subset table pairs and show significant improvement over previous tabular neural models. Third, we present a detailed ablation study to highlight which sketches are crucial for which tasks. Fourth, we use these finetuned models to perform table search; i.e., given a query table, find other tables in a corpus that are unionable, joinable, or that are subsets of the query. Our results demonstrate significant improvements in F1 scores for search compared to state-of-the-art techniques. Finally, we show significant transfer across datasets and tasks establishing that our model can generalize across different tasks and over different data lakes.

The accurate ab initio simulation of molecules and periodic solids with diagrammatic perturbation theory is an important task in quantum chemistry, condensed matter physics, and materials science. In this article, we present the WeakCoupling module of the open-source software package Green, which implements fully self-consistent diagrammatic weak coupling simulations, capable of dealing with real materials in the finite-temperature formalism. The code is licensed under the permissive MIT license. We provide self-consistent GW (scGW) and self-consistent second-order Green's function perturbation theory (GF2) solvers, analysis tools, and post-processing methods. This paper summarizes the theoretical methods implemented and provides background, tutorials and practical instructions for running simulations., Comment: 23 pages, 2 figures

Faults are complex systems embedded in an evolving medium fractured by seismic ruptures. This off-fault damage zone is shown to be thermo-hydro-mechano-chemically coupled to the main fault plane by a growing number of studies. Yet, off-fault medium is still, for the most part, modelled as a purely elastic -- hence passive -- medium. Using a micromechanical model that accounts for dynamic changes of elastic moduli and inelastic strains related to crack growth, we investigate the depth variation of dynamically triggered off-fault damage and its counter-impact on earthquake slip dynamics. We show that the damage zone, while narrowing with depth, also becomes denser and contrary to prevailing assumptions continues to act as an energy sink, significantly influencing rupture dynamics by stabilizing slip rates. Furthermore, we observe that damage formation markedly reduces rupture velocity and delays, or even prevents, the transition to supershear speeds even for a narrow damage zone. This underscores the critical need to incorporate the complex interplay between the main fault plane and its surrounding medium across the entire seismogenic zone. As a proof of concept, we introduce a 1D spring-slider model that captures bulk elastic variations, by modulating spring stiffness, and normal stress variations that emulate changes in bulk load. This simple model demonstrates the occurrence of slow slip events alongside conventional earthquakes, driven by the dynamic interaction between bulk temporal evolution and fault slip dynamics, without necessitating any changes to frictional properties.

In this work, we study a reaction-diffusion system for three species. The nonlinearity in the system comes from an underlying chemical reaction. Our main objective is to understand the long time behaviour of solutions to this reaction-diffusion system when there are degeneracies. More precisely, we treat cases when one of the diffusion coefficients vanishes while the other two diffusion coefficients stay positive. We prove convergence to equilibrium type results. In all our results, the constants appearing in the decay estimates are explicit., Comment: 30 pages, comments welcome

We present a comparison of various approximations to self-consistency in the GW method, including the one-shot G0W0 method, different quasiparticle self-consistency schemes, and the fully self-consistent GW (scGW) approach. To ensure an unbiased and equitable comparison, we have implemented all the schemes with the same underlying Matsubara formalism, while employing Gaussian orbitals to describe the system. Aiming to assess and compare different GW schemes, we analyze band gaps in semiconductors and insulators, as well as ionization potentials in molecules. Our findings reveal that for solids, the different self-consistency schemes perform very similarly. However, for molecules, full self-consistency outperforms all other approximations, i.e., the one-shot and quasiparticle self-consistency GW schemes. Our work highlights the importance of implementation details when comparing different GW methods. By employing state-of-the-art fully self-consistent, finite temperature GW calculations, we have successfully addressed discrepancies in the existing literature regarding its performance. Our results also indicate that when stringent thresholds are imposed, the scGW method consistently yields accurate results., Comment: 16 pages, 5 figures, 3 tables

Over the past decade, there has been a severe staffing shortage in mental healthcare, exacerbated by increased demand for mental health services due to COVID-19. This demand is projected to increase over the next decade or so, necessitating proactive workforce planning to ensure sufficient staffing for ongoing service delivery. Despite the subject's critical significance, the present literature lacks thorough research dedicated to developing a model that addresses the long-term workforce needs required for efficient mental healthcare planning. Furthermore, our interactions with mental health practitioners within the United Kingdom's National Health Service (NHS) revealed the practical need for such a model. To address this gap, we aim to develop a hybrid predictive and prescriptive modelling framework, which combines long-term probabilistic forecasting with an analytical stock-flow model, designed specifically for mental health workforce planning. Given the vital role of nurses, who account for one-third of the total mental health workforce, we focus on modelling the headcount of nurses, but the proposed model can be generalised to other types of workforce planning in the healthcare sector. Using statistical and machine learning approaches and real-world data from NHS, we first identify factors contributing to variations in workforce requirements, then develop a long-term forecasting model to estimate future workforce needs, and finally integrate it into an analytical stock-flow method to provide policy analysis. Our findings highlight the unsustainable nature of present staffing plans, showing a growing nursing shortage. Furthermore, the policy analysis demonstrates the ineffectiveness of blanket remedies, highlighting the need for regional-level policy developments., Comment: Under review

Depression, a prevalent mental health disorder impacting millions globally, demands reliable assessment systems. Unlike previous studies that focus solely on either detecting depression or predicting its severity, our work identifies individual symptoms of depression while also predicting its severity using speech input. We leverage self-supervised learning (SSL)-based speech models to better utilize the small-sized datasets that are frequently encountered in this task. Our study demonstrates notable performance improvements by utilizing SSL embeddings compared to conventional speech features. We compare various types of SSL pretrained models to elucidate the type of speech information (semantic, speaker, or prosodic) that contributes the most in identifying different symptoms. Additionally, we evaluate the impact of combining multiple SSL embeddings on performance. Furthermore, we show the significance of multi-task learning for identifying depressive symptoms effectively., Comment: Accepted at INTERSPEECH 2024

Current automatic depression detection systems provide predictions directly without relying on the individual symptoms/items of depression as denoted in the clinical depression rating scales. In contrast, clinicians assess each item in the depression rating scale in a clinical setting, thus implicitly providing a more detailed rationale for a depression diagnosis. In this work, we make a first step towards using the acoustic features of speech to predict individual items of the depression rating scale before obtaining the final depression prediction. For this, we use convolutional (CNN) and recurrent (long short-term memory (LSTM)) neural networks. We consider different approaches to learning the temporal context of speech. Further, we analyze two variants of voting schemes for individual item prediction and depression detection. We also include an animated visualization that shows an example of item prediction over time as the speech progresses.

Pulsar timing array experiments have recently uncovered evidence for a nanohertz gravitational wave background by precisely timing an ensemble of millisecond pulsars. The next significant milestones for these experiments include characterizing the detected background with greater precision, identifying its source(s), and detecting continuous gravitational waves from individual supermassive black hole binaries. To achieve these objectives, generating accurate and precise times of arrival of pulses from pulsar observations is crucial. Incorrect polarization calibration of the observed pulsar profiles may introduce errors in the measured times of arrival. Further, previous studies (e.g., van Straten 2013; Manchester et al. 2013) have demonstrated that robust polarization calibration of pulsar profiles can reduce noise in the pulsar timing data and improve timing solutions. In this paper, we investigate and compare the impact of different polarization calibration methods on pulsar timing precision using three distinct calibration techniques: the Ideal Feed Assumption (IFA), Measurement Equation Modeling (MEM), and Measurement Equation Template Matching (METM). Three NANOGrav pulsars-PSRs J1643$-$1224, J1744$-$1134, and J1909$-$3744-observed with the 800 MHz and 1.5 GHz receivers at the Green Bank Telescope (GBT) are utilized for our analysis. Our findings reveal that all three calibration methods enhance timing precision compared to scenarios where no polarization calibration is performed. Additionally, among the three calibration methods, the IFA approach generally provides the best results for timing analysis of pulsars observed with the GBT receiver system. We attribute the comparatively poorer performance of the MEM and METM methods to potential instabilities in the reference noise diode coupled to the receiver and temporal variations in the profile of the reference pulsar, respectively., Comment: 21 pages, 6 figures, 1 table, submitted to Astrophysical Journal

Despite their remarkable successes, state-of-the-art large language models (LLMs), including vision-and-language models (VLMs) and unimodal language models (ULMs), fail to understand precise semantics. For example, semantically equivalent sentences expressed using different lexical compositions elicit diverging representations. The degree of this divergence and its impact on encoded semantics is not very well understood. In this paper, we introduce the SUGARCREPE++ dataset to analyze the sensitivity of VLMs and ULMs to lexical and semantic alterations. Each sample in SUGARCREPE++ dataset consists of an image and a corresponding triplet of captions: a pair of semantically equivalent but lexically different positive captions and one hard negative caption. This poses a 3-way semantic (in)equivalence problem to the language models. We comprehensively evaluate VLMs and ULMs that differ in architecture, pre-training objectives and datasets to benchmark the performance of SUGARCREPE++ dataset. Experimental results highlight the difficulties of VLMs in distinguishing between lexical and semantic variations, particularly in object attributes and spatial relations. Although VLMs with larger pre-training datasets, model sizes, and multiple pre-training objectives achieve better performance on SUGARCREPE++, there is a significant opportunity for improvement. We show that all the models which achieve better performance on compositionality datasets need not perform equally well on SUGARCREPE++, signifying that compositionality alone may not be sufficient for understanding semantic and lexical alterations. Given the importance of the property that the SUGARCREPE++ dataset targets, it serves as a new challenge to the vision-and-language community., Comment: Added the dataset link to the abstract

As Large Language Models (LLMs) are increasingly being employed in real-world applications in critical domains such as healthcare, it is important to ensure that the Chain-of-Thought (CoT) reasoning generated by these models faithfully captures their underlying behavior. While LLMs are known to generate CoT reasoning that is appealing to humans, prior studies have shown that these explanations do not accurately reflect the actual behavior of the underlying LLMs. In this work, we explore the promise of three broad approaches commonly employed to steer the behavior of LLMs to enhance the faithfulness of the CoT reasoning generated by LLMs: in-context learning, fine-tuning, and activation editing. Specifically, we introduce novel strategies for in-context learning, fine-tuning, and activation editing aimed at improving the faithfulness of the CoT reasoning. We then carry out extensive empirical analyses with multiple benchmark datasets to explore the promise of these strategies. Our analyses indicate that these strategies offer limited success in improving the faithfulness of the CoT reasoning, with only slight performance enhancements in controlled scenarios. Activation editing demonstrated minimal success, while fine-tuning and in-context learning achieved marginal improvements that failed to generalize across diverse reasoning and truthful question-answering benchmarks. In summary, our work underscores the inherent difficulty in eliciting faithful CoT reasoning from LLMs, suggesting that the current array of approaches may not be sufficient to address this complex challenge.

We present a novel method for extracting neural embeddings that model the background acoustics of a speech signal. The extracted embeddings are used to estimate specific parameters related to the background acoustic properties of the signal in a non-intrusive manner, which allows the embeddings to be explainable in terms of those parameters. We illustrate the value of these embeddings by performing clustering experiments on unseen test data and show that the proposed embeddings achieve a mean F1 score of 95.2\% for three different tasks, outperforming significantly the WavLM based signal embeddings. We also show that the proposed method can explain the embeddings by estimating 14 acoustic parameters characterizing the background acoustics, including reverberation and noise levels, overlapped speech detection, CODEC type detection and noise type detection with high accuracy and a real-time factor 17 times lower than an external baseline method.

The success of iterative pruning methods in achieving state-of-the-art sparse networks has largely been attributed to improved mask identification and an implicit regularization induced by pruning. We challenge this hypothesis and instead posit that their repeated cyclic training schedules enable improved optimization. To verify this, we show that pruning at initialization is significantly boosted by repeated cyclic training, even outperforming standard iterative pruning methods. The dominant mechanism how this is achieved, as we conjecture, can be attributed to a better exploration of the loss landscape leading to a lower training loss. However, at high sparsity, repeated cyclic training alone is not enough for competitive performance. A strong coupling between learnt parameter initialization and mask seems to be required. Standard methods obtain this coupling via expensive pruning-training iterations, starting from a dense network. To achieve this with sparse training instead, we propose SCULPT-ing, i.e., repeated cyclic training of any sparse mask followed by a single pruning step to couple the parameters and the mask, which is able to match the performance of state-of-the-art iterative pruning methods in the high sparsity regime at reduced computational cost.

Adaptive Boosting with Dynamic Weight Adjustment is an enhancement of the traditional Adaptive boosting commonly known as AdaBoost, a powerful ensemble learning technique. Adaptive Boosting with Dynamic Weight Adjustment technique improves the efficiency and accuracy by dynamically updating the weights of the instances based on prediction error where the weights are updated in proportion to the error rather than updating weights uniformly as we do in traditional Adaboost. Adaptive Boosting with Dynamic Weight Adjustment performs better than Adaptive Boosting as it can handle more complex data relations, allowing our model to handle imbalances and noise better, leading to more accurate and balanced predictions. The proposed model provides a more flexible and effective approach for boosting, particularly in challenging classification tasks.

Theoretical descriptions of non equilibrium dynamics of quantum many-body systems essentially employ either (i) explicit treatments, relying on truncation of the expansion of the many-body wave function, (ii) compressed representations of the many-body wave function, or (iii) evolution of an effective (downfolded) representation through Green's functions. In this work, we select representative cases of each of the methods and address how these complementary approaches capture the dynamics driven by intense field perturbations to non equilibrium states. Under strong driving, the systems are characterized by strong entanglement of the single particle density matrix and natural populations approaching those of a strongly interacting equilibrium system. We generate a representative set of results that are numerically exact and form a basis for critical comparison of the distinct families of methods. We demonstrate that the compressed formulation based on similarity transformed Hamiltonians (coupled cluster approach) is practically exact in weak fields and, hence, weakly or moderately correlated systems. Coupled cluster, however, struggles for strong driving fields, under which the system exhibits strongly correlated behavior, as measured by the von Neumann entropy of the single particle density matrix. The dynamics predicted by Green's functions in the (widely popular) GW approximation are less accurate by improve significantly upon the mean-field results in the strongly driven regime., Comment: 11 pages, 2 figures in main text. 11 page supplemental information with 6 figures

Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this work, we propose PathoLogy Universal TransfOrmer (PLUTO): a light-weight pathology FM that is pre-trained on a diverse dataset of 195 million image tiles collected from multiple sites and extracts meaningful representations across multiple WSI scales that enable a large variety of downstream pathology tasks. In particular, we design task-specific adaptation heads that utilize PLUTO's output embeddings for tasks which span pathology scales ranging from subcellular to slide-scale, including instance segmentation, tile classification, and slide-level prediction. We compare PLUTO's performance to other state-of-the-art methods on a diverse set of external and internal benchmarks covering multiple biologically relevant tasks, tissue types, resolutions, stains, and scanners. We find that PLUTO matches or outperforms existing task-specific baselines and pathology-specific foundation models, some of which use orders-of-magnitude larger datasets and model sizes when compared to PLUTO. Our findings present a path towards a universal embedding to power pathology image analysis, and motivate further exploration around pathology foundation models in terms of data diversity, architectural improvements, sample efficiency, and practical deployability in real-world applications.

Recent breakthroughs in large models have highlighted the critical significance of data scale, labels and modals. In this paper, we introduce MS MARCO Web Search, the first large-scale information-rich web dataset, featuring millions of real clicked query-document labels. This dataset closely mimics real-world web document and query distribution, provides rich information for various kinds of downstream tasks and encourages research in various areas, such as generic end-to-end neural indexer models, generic embedding models, and next generation information access system with large language models. MS MARCO Web Search offers a retrieval benchmark with three web retrieval challenge tasks that demand innovations in both machine learning and information retrieval system research domains. As the first dataset that meets large, real and rich data requirements, MS MARCO Web Search paves the way for future advancements in AI and system research. MS MARCO Web Search dataset is available at: https://github.com/microsoft/MS-MARCO-Web-Search., Comment: 10 pages, 6 figures, for associated dataset, see http://github.com/microsoft/MS-MARCO-Web-Search

Modern cloud has turned data services into easily accessible commodities. With just a few clicks, users are now able to access a catalog of data processing systems for a wide range of tasks. However, the cloud brings in both complexity and opportunity. While cloud users can quickly start an application by using various data services, it can be difficult to configure and optimize these services to gain the most value from them. For cloud providers, managing every aspect of an ever-increasing set of data services, while meeting customer SLAs and minimizing operational cost is becoming more challenging. Cloud technology enables the collection of significant amounts of workload traces and system telemetry. With the progress in data science (DS) and machine learning (ML), it is feasible and desirable to utilize a data-driven, ML-based approach to automate various aspects of data services, resulting in the creation of autonomous data services. This paper presents our perspectives and insights on creating autonomous data services on Azure. It also covers the future endeavors we plan to undertake and unresolved issues that still need attention., Comment: SIGMOD Companion of the 2023 International Conference on Management of Data. 2023

In this paper, we derive maximal acceleration of a massive particle in Rainbow gravity. Using eight-dimensional phase-space metric compatible with Rainbow gravity, we obtain the maximal acceleration, valid up to first order in the Rainbow gravity parameter $\eta$. Using the positivity condition on maximal acceleration, we find the upper bound on the Rainbow gravity parameter is of the order of $~10^{22}$ for positron and $10^{-44}$ for a black hole. After obtaining the expression for maximal acceleration for different choices of Rainbow functions, we derive corresponding modifications to Unruh temperature. Comparing with the observational value of the Unruh temperature, we find the upper bound on $\eta$ as $~10^{32}$ for positron radiation. %and of the order of $10^{-100}$ for radiation from a black hole. We then derive geodesic equations for different choices of Rainbow functions and also obtain Newtonian limit of these geodesic equations. We find that the changes in the value of maximum acceleration, maximum temperature and Newtonian force equation are dependent on the choices of Rainbow functions., Comment: 12 pages

Neural Radiance Fields (NeRF) show impressive performance in photo-realistic free-view rendering of scenes. Recent improvements on the NeRF such as TensoRF and ZipNeRF employ explicit models for faster optimization and rendering, as compared to the NeRF that employs an implicit representation. However, both implicit and explicit radiance fields require dense sampling of images in the given scene. Their performance degrades significantly when only a sparse set of views is available. Researchers find that supervising the depth estimated by a radiance field helps train it effectively with fewer views. The depth supervision is obtained either using classical approaches or neural networks pre-trained on a large dataset. While the former may provide only sparse supervision, the latter may suffer from generalization issues. As opposed to the earlier approaches, we seek to learn the depth supervision by designing augmented models and training them along with the main radiance field. Further, we aim to design a framework of regularizations that can work across different implicit and explicit radiance fields. We observe that certain features of these radiance field models overfit to the observed images in the sparse-input scenario. Our key finding is that reducing the capability of the radiance fields with respect to positional encoding, the number of decomposed tensor components or the size of the hash table, constrains the model to learn simpler solutions, which estimate better depth in certain regions. By designing augmented models based on such reduced capabilities, we obtain better depth supervision for the main radiance field. We achieve state-of-the-art view-synthesis performance with sparse input views on popular datasets containing forward-facing and 360$^\circ$ scenes by employing the above regularizations., Comment: The source code for our model can be found on our project page: https://nagabhushansn95.github.io/publications/2024/Simple-RF.html. arXiv admin note: substantial text overlap with arXiv:2309.03955