Search

Time series forecasts are often influenced by exogenous contextual features in addition to their corresponding history. For example, in financial settings, it is hard to accurately predict a stock price without considering public sentiments and policy decisions in the form of news articles, tweets, etc. Though this is common knowledge, the current state-of-the-art (SOTA) forecasting models fail to incorporate such contextual information, owing to its heterogeneity and multimodal nature. To address this, we introduce ContextFormer, a novel plug-and-play method to surgically integrate multimodal contextual information into existing pre-trained forecasting models. ContextFormer effectively distills forecast-specific information from rich multimodal contexts, including categorical, continuous, time-varying, and even textual information, to significantly enhance the performance of existing base forecasters. ContextFormer outperforms SOTA forecasting models by up to 30% on a range of real-world datasets spanning energy, traffic, environmental, and financial domains.

Having a detailed knowledge of the intracluster medium (ICM) to infer the exact cluster physics such as the cluster dynamical state is crucial for cluster-based cosmological studies. This knowledge limits the accuracy and precision of mass estimation, a key parameter for such studies. In this paper, we conduct an in-depth analysis of cluster ACT-CL J0240.0+0116 using a multi-wavelength approach, with a primary focus on high angular resolution Sunyaev-Zeldovich (SZ) thermal component observations obtained under the NIKA2 Sunyaev-Zeldovich Large Programme (LPSZ). We create composite images using NIKA2, X-ray, and optical galaxy number density maps. The results reveal distinct signs of disturbance within the cluster with the distributions of gas and member galaxies that do not overlap. We also find suggestions of an inflow of matter onto the cluster from the southwestern direction. Ultimately, we classify the cluster as disturbed, using morphological indicators derived from its SZ, X-ray, and optical image. The cluster SZ signal is also contaminated by a strong central point source. We adopt different approaches to handling this contaminant and find the estimates of our pressure and hydrostatic mass profiles robust to the point source mitigation model. The cluster hydrostatic mass is estimated at $4.25^{+0.50}_{-0.45\, } \times 10^{14} \,\mathrm{M}_{\odot}$ for the case where the point source was masked. These values are consistent with the mass estimated using only X-ray data and with those from previous SZ studies of the Atacama cosmology telescope (ACT) survey, with improved precision on the mass estimate. Our findings strongly suggest that ACT-CL J0240.0+0116 is a disturbed cluster system, and the detailed observations and derived values serve as a compelling case study for the capabilities of the LPSZ in mapping the cluster ICM with high precision., Comment: 13 pages, 7 figures, submitted to A&A

While style transfer techniques have been well-developed for 2D image stylization, the extension of these methods to 3D scenes remains relatively unexplored. Existing approaches demonstrate proficiency in transferring colors and textures but often struggle with replicating the geometry of the scenes. In our work, we leverage an explicit Gaussian Splatting (GS) representation and directly match the distributions of Gaussians between style and content scenes using the Earth Mover's Distance (EMD). By employing the entropy-regularized Wasserstein-2 distance, we ensure that the transformation maintains spatial smoothness. Additionally, we decompose the scene stylization problem into smaller chunks to enhance efficiency. This paradigm shift reframes stylization from a pure generative process driven by latent space losses to an explicit matching of distributions between two Gaussian representations. Our method achieves high-resolution 3D stylization by faithfully transferring details from 3D style scenes onto the content scene. Furthermore, WaSt-3D consistently delivers results across diverse content and style scenes without necessitating any training, as it relies solely on optimization-based techniques. See our project page for additional results and source code: $\href{https://compvis.github.io/wast3d/}{https://compvis.github.io/wast3d/}$.

Ranking a set of items based on their relevance to a given query is a core problem in search and recommendation. Transformer-based ranking models are the state-of-the-art approaches for such tasks, but they score each query-item independently, ignoring the joint context of other relevant items. This leads to sub-optimal ranking accuracy and high computational costs. In response, we propose Cross-encoders with Joint Efficient Modeling (CROSS-JEM), a novel ranking approach that enables transformer-based models to jointly score multiple items for a query, maximizing parameter utilization. CROSS-JEM leverages (a) redundancies and token overlaps to jointly score multiple items, that are typically short-text phrases arising in search and recommendations, and (b) a novel training objective that models ranking probabilities. CROSS-JEM achieves state-of-the-art accuracy and over 4x lower ranking latency over standard cross-encoders. Our contributions are threefold: (i) we highlight the gap between the ranking application's need for scoring thousands of items per query and the limited capabilities of current cross-encoders; (ii) we introduce CROSS-JEM for joint efficient scoring of multiple items per query; and (iii) we demonstrate state-of-the-art accuracy on standard public datasets and a proprietary dataset. CROSS-JEM opens up new directions for designing tailored early-attention-based ranking models that incorporate strict production constraints such as item multiplicity and latency.

In Sunyaev-Zeldovich (SZ) cluster cosmology, two tools are needed to be able to exploit data from large scale surveys in the millimeter-wave domain. An accurate description of the IntraCluster Medium (ICM) pressure profile is needed along with the scaling relation connecting the SZ brightness to the mass. With its high angular resolution and large field of view, The NIKA2 camera, operating at 150 and 260 GHz, is perfectly suited for precise cluster SZ mapping. The SZ Large Program (LPSZ) of the NIKA2 collaboration is dedicated to the observation of a sample of 38 SZ-selected clusters at intermediate to high redshift and observed both in SZ and X-ray. The current status is that all LPSZ clusters have been observed and the analysis toward the final results is ongoing. We present in detail how NIKA2-LPSZ will obtain a robust estimation of the SZ-Mass scaling relation and how it will be used to obtain cosmological constraints., Comment: 4 pages, 3 figures, contribution to the 2024 Cosmology session of the 58th Rencontres de Moriond

Concrete manufacturing projects are one of the most common ones for consulting agencies. Because of the highly non-linear dependency of input materials like ash, water, cement, superplastic, etc; with the resultant strength of concrete, it gets difficult for machine learning models to successfully capture this relation and perform cost optimizations. This paper highlights how PINNs (Physics Informed Neural Networks) can be useful in the given situation. This state-of-the-art model shall also get compared with traditional models like Linear Regression, Random Forest, Gradient Boosting, and Deep Neural Network. Results of the research highlights how well PINNs performed even with reduced dataset, thus resolving one of the biggest issues of limited data availability for ML models. On an average, PINN got the loss value reduced by 26.3% even with 40% lesser data compared to the Deep Neural Network. In addition to predicting strength of the concrete given the quantity of raw materials, the paper also highlights the use of heuristic optimization method like Particle Swarm Optimization (PSO) in predicting quantity of raw materials required to manufacture concrete of given strength with least cost.

In this work, we present a GPU-accelerated library for the underlying components of Kolmogorov-Arnold Networks (KANs), along with an algorithm to eliminate bounded grids in KANs. The GPU-accelerated library reduces the computational complexity of Basis Spline (B-spline) evaluation by a factor of $\mathcal{O}$(grid size) compared to existing codes, enabling batch computation for large-scale learning. To overcome the limitations of traditional KANs, we introduce Unbounded KANs (UKANs), which eliminate the need for a bounded grid and a fixed number of B-spline coefficients. To do so, we replace the KAN parameters (B-spline coefficients) with a coefficient generator (CG) model. The inputs to the CG model are designed based on the idea of an infinite symmetric grid extending from negative infinity to positive infinity. The positional encoding of grid group, a sequential collection of B-spline grid indexes, is fed into the CG model, and coefficients are consumed by the efficient implementation (matrix representations) of B-spline functions to generate outputs. We perform several experiments on regression, classification, and generative tasks, which are promising. In particular, UKAN does not require data normalization or a bounded domain for evaluation. Additionally, our benchmarking results indicate the superior memory and computational efficiency of our library compared to existing codes., Comment: 10 pages, 7 figures, 5 tables

The recent rise of CubeSat has revolutionized global space explorations, as it offers cost-effective solutions for low-orbit space applications (including climate monitoring, weather measurements, communications, and earth observation). A salient feature of CubeSat is that applications currently on-boarded can either be updated or entirely replaced by new applications via software updates, which allows reusing in-orbit hardware, reduces space debris, and saves cost as well as time. Securing software updates employing traditional methods (e.g., encryption) remains impractical mainly due to the low-resource capabilities of CubeSat. Therefore, the security of software updates for CubeSats remains a critical issue. In this paper, we propose CubeSat Update Mechanism (CSUM), a lightweight scheme to provide integrity, authentication, and data freshness guarantees for software update broadcasts to CubeSats using a hash chain. We empirically evaluate our proof of concept implementation to demonstrate the feasibility and effectiveness of our approach. CSUM can validate 50,000 consecutive updates successfully in less than a second. We also perform a comparative analysis of different cryptographic primitives. Our empirical evaluations show that the hash-based approach is at least 61$\times$ faster than the conventional mechanisms, even in resource-constrained environments. Finally, we discuss the limitations, challenges, and potential future research directions for CubeSat software update procedures., Comment: This is an extended version of our paper accepted at IEEE LCN 2024

Accurate prediction of protein-ligand binding affinities is crucial for drug development. Recent advances in machine learning show promising results on this task. However, these methods typically rely heavily on labeled data, which can be scarce or unreliable, or they rely on assumptions like Boltzmann-distributed data that may not hold true in practice. Here, we present DualBind, a novel framework that integrates supervised mean squared error (MSE) with unsupervised denoising score matching (DSM) to accurately learn the binding energy function. DualBind not only addresses the limitations of DSM-only models by providing more accurate absolute affinity predictions but also improves generalizability and reduces reliance on labeled data compared to MSE-only models. Our experimental results demonstrate that DualBind excels in predicting binding affinities and can effectively utilize both labeled and unlabeled data to enhance performance., Comment: Preprint, work in progress

Neurons in the brain continuously process the barrage of sensory inputs they receive from the environment. A wide array of experimental work has shown that the collective activity of neural populations encodes and processes this constant bombardment of information. How these collective patterns of activity depend on single neuron properties is often unclear. Single-neuron recordings have shown that individual neural responses to inputs are nonlinear, which prevents a straightforward extrapolation from single neuron features to emergent collective states. In this work, we use a field theoretic formulation of a stochastic leaky integrate-and-fire model to study the impact of nonlinear intensity functions on macroscopic network activity. We show that the interplay between nonlinear spike emission and membrane potential resets can i) give rise to metastable transitions between active firing rate states, and ii) can enhance or suppress mean firing rates and membrane potentials in opposite directions.

While there are a plethora of methods for link prediction in knowledge graphs, state-of-the-art approaches are often black box, obfuscating model reasoning and thereby limiting the ability of users to make informed decisions about model predictions. Recently, methods have emerged to generate prediction explanations for Knowledge Graph Embedding models, a widely-used class of methods for link prediction. The question then becomes, how well do these explanation systems work? To date this has generally been addressed anecdotally, or through time-consuming user research. In this work, we present an in-depth exploration of a simple link prediction explanation method we call LinkLogic, that surfaces and ranks explanatory information used for the prediction. Importantly, we construct the first-ever link prediction explanation benchmark, based on family structures present in the FB13 dataset. We demonstrate the use of this benchmark as a rich evaluation sandbox, probing LinkLogic quantitatively and qualitatively to assess the fidelity, selectivity and relevance of the generated explanations. We hope our work paves the way for more holistic and empirical assessment of knowledge graph prediction explanation methods in the future., Comment: 12 pages, 4 figures in main text. For code and data, see https://github.com/niraj17singh/LinkLogic

Robotic Process Automation (RPA) systems face challenges in handling complex processes and diverse screen layouts that require advanced human-like decision-making capabilities. These systems typically rely on pixel-level encoding through drag-and-drop or automation frameworks such as Selenium to create navigation workflows, rather than visual understanding of screen elements. In this context, we present SmartFlow, an AI-based RPA system that uses pre-trained large language models (LLMs) coupled with deep-learning based image understanding. Our system can adapt to new scenarios, including changes in the user interface and variations in input data, without the need for human intervention. SmartFlow uses computer vision and natural language processing to perceive visible elements on the graphical user interface (GUI) and convert them into a textual representation. This information is then utilized by LLMs to generate a sequence of actions that are executed by a scripting engine to complete an assigned task. To assess the effectiveness of SmartFlow, we have developed a dataset that includes a set of generic enterprise applications with diverse layouts, which we are releasing for research use. Our evaluations on this dataset demonstrate that SmartFlow exhibits robustness across different layouts and applications. SmartFlow can automate a wide range of business processes such as form filling, customer service, invoice processing, and back-office operations. SmartFlow can thus assist organizations in enhancing productivity by automating an even larger fraction of screen-based workflows. The demo-video and dataset are available at https://smartflow-4c5a0a.webflow.io/., Comment: 32nd ACM International Conference on Information and Knowledge Management

Creative story illustration requires a consistent interplay of multiple characters or objects. However, conventional text-to-image models face significant challenges while producing images featuring multiple personalized subjects. For example, they distort the subject rendering, or the text descriptions fail to render coherent subject interactions. We present Multi-Subject Personalization (MSP) to alleviate some of these challenges. We implement MSP using Stable Diffusion and assess our approach against other text-to-image models, showcasing its consistent generation of good-quality images representing intended subjects and interactions., Comment: 2023 Conference on Neural Information Processing Systems

Textual image generation spans diverse fields like advertising, education, product packaging, social media, information visualization, and branding. Despite recent strides in language-guided image synthesis using diffusion models, current models excel in image generation but struggle with accurate text rendering and offer limited control over font attributes. In this paper, we aim to enhance the synthesis of high-quality images with precise text customization, thereby contributing to the advancement of image generation models. We call our proposed method CustomText. Our implementation leverages a pre-trained TextDiffuser model to enable control over font color, background, and types. Additionally, to address the challenge of accurately rendering small-sized fonts, we train the ControlNet model for a consistency decoder, significantly enhancing text-generation performance. We assess the performance of CustomText in comparison to previous methods of textual image generation on the publicly available CTW-1500 dataset and a self-curated dataset for small-text generation, showcasing superior results., Comment: Accepted by AI for Content Creation (AI4CC) workshop at CVPR 2024

In this position paper, we explore the power of storytelling and its connection to place through the use of Augmented Reality (AR) technology, particularly within the context of Th\'amien Ohlone history on the Santa Clara University campus. To do this, we utilized SLAM and 8th Wall to create virtual, location-based experiences that geolocate tribal stories at present-day sites, showcase the living culture of the Th\'amien Ohlone tribe, and advocate for physical markers that could exist to recognize their story. When doing so, we made sure to select locations that added to the story each stop tells to serve as our anchors. Our research then investigates both the social and technical considerations involved in selecting anchors for AR experiences, using the Th\'amien Ohlone AR Tour as a case study., Comment: Presented at CHI 2024 (arXiv:2404.05889)

The field of 3D reconstruction from images has rapidly evolved in the past few years, first with the introduction of Neural Radiance Field (NeRF) and more recently with 3D Gaussian Splatting (3DGS). The latter provides a significant edge over NeRF in terms of the training and inference speed, as well as the reconstruction quality. Although 3DGS works well for dense input images, the unstructured point-cloud like representation quickly overfits to the more challenging setup of extremely sparse input images (e.g., 3 images), creating a representation that appears as a jumble of needles from novel views. To address this issue, we propose regularized optimization and depth-based initialization. Our key idea is to introduce a structured Gaussian representation that can be controlled in 2D image space. We then constraint the Gaussians, in particular their position, and prevent them from moving independently during optimization. Specifically, we introduce single and multiview constraints through an implicit convolutional decoder and a total variation loss, respectively. With the coherency introduced to the Gaussians, we further constrain the optimization through a flow-based loss function. To support our regularized optimization, we propose an approach to initialize the Gaussians using monocular depth estimates at each input view. We demonstrate significant improvements compared to the state-of-the-art sparse-view NeRF-based approaches on a variety of scenes., Comment: Project page: https://people.engr.tamu.edu/nimak/Papers/CoherentGS

An accurate reconstruction of galaxy cluster masses is key to use this population of objects as a cosmological probe. In this work we present a study on the hydrostatic-to-lensing mass scaling relation for a sample of 53 clusters whose masses were reconstructed homogeneously in a redshift range between $z= 0.05$ and $1.07$. The $M_{500}$ mass for each cluster was indeed inferred from the mass profiles extracted from the X-ray and lensing data, without using a priori observable-mass scaling relations. We assessed the systematic dispersion of the masses estimated with our reference analyses with respect to other published mass estimates. Accounting for this systematic scatter does not change our main results, but enables the propagation of the uncertainties related to the mass reconstruction method or used dataset. Our analysis gives a hydrostatic-to-lensing mass bias of $(1-b) =0.739^{+0.075}_{-0.070}$ and no evidence of evolution with redshift. These results are robust against possible subsample differences.