Search

Multimodal Large Language Models (MLLMs) have driven major advances in video understanding, yet their vulnerability to adversarial tampering and manipulations remains underexplored. To address this gap, we introduce MVTamperBench, a benchmark that systematically evaluates MLLM robustness against five prevalent tampering techniques: rotation, masking, substitution, repetition, and dropping. Built from 3.4K original videos-expanded to over 17K tampered clips spanning 19 video tasks. MVTamperBench challenges models to detect manipulations in spatial and temporal coherence. We evaluate 45 recent MLLMs from 15+ model families, revealing substantial variability in resilience across tampering types and showing that larger parameter counts do not necessarily guarantee robustness. MVTamperBench sets a new benchmark for developing tamper-resilient MLLM in safety-critical applications, including detecting clickbait, preventing harmful content distribution, and enforcing policies on media platforms. We release all code and data to foster open research in trustworthy video understanding. Code: https://amitbcp.github.io/MVTamperBench/ Data: https://huggingface.co/datasets/Srikant86/MVTamperBench

As businesses increasingly rely on automation to streamline operations, the limitations of Robotic Process Automation (RPA) have become apparent, particularly its dependence on expert knowledge and inability to handle complex decision-making tasks. Recent advancements in Artificial Intelligence (AI), particularly Generative AI (GenAI) and Large Language Models (LLMs), have paved the way for Intelligent Automation (IA), which integrates cognitive capabilities to overcome the shortcomings of RPA. This paper introduces Text2Workflow, a novel method that automatically generates workflows from natural language user requests. Unlike traditional automation approaches, Text2Workflow offers a generalized solution for automating any business process, translating user inputs into a sequence of executable steps represented in JavaScript Object Notation (JSON) format. Leveraging the decision-making and instruction-following capabilities of LLMs, this method provides a scalable, adaptable framework that enables users to visualize and execute workflows with minimal manual intervention. This research outlines the Text2Workflow methodology and its broader implications for automating complex business processes., Comment: Under review at Elsevier's Engineering Applications of Artificial Intelligence

In contrast to non-medical image denoising, where enhancing image clarity is the primary goal, medical image denoising warrants preservation of crucial features without introduction of new artifacts. However, many denoising methods that improve the clarity of the image, inadvertently alter critical information of the denoised images, potentially compromising classification performance and diagnostic quality. Additionally, supervised denoising methods are not very practical in medical image domain, since a \emph{ground truth} denoised version of a noisy medical image is often extremely challenging to obtain. In this paper, we tackle both of these problems by introducing a novel neural network based method -- \emph{Contextual Checkerboard Denoising}, that can learn denoising from only a dataset of noisy images, while preserving crucial anatomical details necessary for image classification/analysis. We perform our experimentation on real Optical Coherence Tomography (OCT) images, and empirically demonstrate that our proposed method significantly improves image quality, providing clearer and more detailed OCT images, while enhancing diagnostic accuracy., Comment: Under review in Springer Journal of Medical Systems. Code available: https://github.com/AbtahiMajeed/CheckerBoardDenoiser/tree/main

Edge AI computing boxes are a new class of computing devices that are aimed to revolutionize the AI industry. These compact and robust hardware units bring the power of AI processing directly to the source of data--on the edge of the network. On the other hand, on-demand serverless inference services are becoming more and more popular as they minimize the infrastructural cost associated with hosting and running DNN models for small to medium-sized businesses. However, these computing devices are still constrained in terms of resource availability. As such, the service providers need to load and unload models efficiently in order to meet the growing demand. In this paper, we introduce FusedInf to efficiently swap DNN models for on-demand serverless inference services on the edge. FusedInf combines multiple models into a single Direct Acyclic Graph (DAG) to efficiently load the models into the GPU memory and make execution faster. Our evaluation of popular DNN models showed that creating a single DAG can make the execution of the models up to 14\% faster while reducing the memory requirement by up to 17\%. The prototype implementation is available at https://github.com/SifatTaj/FusedInf.

This article addresses an innovative concept to enhance the security for IoT applications in the case of Simultaneous Wireless Information and Power Transfer. This is achieved by integrating a complementary security and identification mechanism through Wireless Power Transfer link within the network of autonomous wireless nodes. This mechanism is implemented at the level of the RF rectifier used to receive energy from a dedicated RF source. A prototype of such RF rectifier has been developed, it generates in real time a backscattered waveform (uplink from the wireless node to the RF source) as function of the parameters of the incoming energy waveform (downlink from RF source) and a digital private key code, generated/available at the level of the wireless node. This uplink waveform can be monitored at the level of the RF source for security/identification purposes implementing an autonomous hardware/physical security layer that operates independently from the communication protocols.

Reliable facial expression learning (FEL) involves the effective learning of distinctive facial expression characteristics for more reliable, unbiased and accurate predictions in real-life settings. However, current systems struggle with FEL tasks because of the variance in people's facial expressions due to their unique facial structures, movements, tones, and demographics. Biased and imbalanced datasets compound this challenge, leading to wrong and biased prediction labels. To tackle these, we introduce GReFEL, leveraging Vision Transformers and a facial geometry-aware anchor-based reliability balancing module to combat imbalanced data distributions, bias, and uncertainty in facial expression learning. Integrating local and global data with anchors that learn different facial data points and structural features, our approach adjusts biased and mislabeled emotions caused by intra-class disparity, inter-class similarity, and scale sensitivity, resulting in comprehensive, accurate, and reliable facial expression predictions. Our model outperforms current state-of-the-art methodologies, as demonstrated by extensive experiments on various datasets., Comment: ACCV 2024. Extended version of ARBEx (arXiv:2305.01486)

This paper introduces and investigates the concept of the $q$-numerical range for tuples of bounded linear operators in Hilbert spaces. We establish various inequalities concerning the $q$-numerical radius associated with these operator tuples. Furthermore, we extend our study to define the $q$-numerical range in semi-Hilbert spaces and provide a proof of its convexity. Additionally, we explore several related results in this context., Comment: 21 pages

A novel method was developed to detect double-$\Lambda$ hypernuclear events in nuclear emulsions using machine learning techniques. The object detection model, the Mask R-CNN, was trained using images generated by Monte Carlo simulations, image processing, and image-style transformation based on generative adversarial networks. Despite being exclusively trained on $\prescript{6\ }{\Lambda\Lambda}{\rm{He}}$ events, the model achieved a detection efficiency of 93.8$\%$ for $\prescript{6\ }{\Lambda\Lambda}{\rm{He}}$ and 82.0$\%$ for $\prescript{5\ }{\Lambda\Lambda}{\rm{H}}$ events in the produced images. In addition, the model demonstrated its ability to detect the $\prescript{6\ }{\Lambda\Lambda}{\rm{He}}$ event named the Nagara event, which is the only uniquely identified double-$\Lambda$ hypernuclear event reported to date. It also exhibited a proper segmentation of the event topology. Furthermore, after analyzing 0.2$\%$ of the entire emulsion data from the J-PARC E07 experiment utilizing the developed approach, six new candidates for double-$\Lambda$ hypernuclear events were detected, suggesting that more than 2000 double-strangeness hypernuclear events were recorded in the entire dataset. This method is sufficiently effective for mining more latent double-$\Lambda$ hypernuclear events recorded in nuclear emulsion sheets by reducing the time required for manual visual inspection by a factor of five hundred.

We prove that there exists a one-to-one correspondence between smooth quartic surfaces with an outer Galois point and K3 surfaces with a certain automorphism of order 4. Furthermore, we characterize quartic surfaces with two or more outer Galois points as K3 surfaces., Comment: 6 pages

In recent literature, matrix product (MP) codes and their duals have gained significant attention due to their application in the construction of quantum stabilizer codes. In this paper, we begin with providing a formula that characterizes the Galois dual of MP codes. Using this formula, we establish the conditions under which MP codes are self-orthogonal and dual-containing. Although similar results may exist in the literature, the novelty and superiority of our results can be identified in the following points. Previous results that characterize the duals of MP codes only apply to MP codes with an invertible square defining matrix $\mathcal{A}$. However, our characterization applies to MP code with any defining matrix, whether $\mathcal{A}$ is not square or not of full row rank. Previous studies on the conditions for self-orthogonality or dual-containment of MP codes have assumed certain structures for the product $\mathcal{A}\mathcal{A}^T$ or $\mathcal{A}\mathcal{A}^{\dagger}$, such as being diagonal, anti-diagonal, monomial, or partitioned Hermitian orthogonal. However, our conditions do not necessitate such specific structures. Previous studies investigated MP code duality in the context of Euclidean and Hermitian duals; however, we investigate MP code duality in the broader context of Galois dual, with Euclidean and Hermitian duals acting as special cases. Finally, it is worth noting that the proposed conditions for Galois self-orthogonality or dual-containment are both necessary and sufficient. To demonstrate the theoretical results, several numerical examples with best-known parameters MP codes are provided.

Nowadays, cloud-based services are widely favored over the traditional approach of locally training a Neural Network (NN) model. Oftentimes, a cloud service processes multiple requests from users--thus training multiple NN models concurrently. However, training NN models concurrently is a challenging process, which typically requires significant amounts of available computing resources and takes a long time to complete. In this paper, we present UnifiedNN to effectively train multiple NN models concurrently on the cloud. UnifiedNN effectively "combines" multiple NN models and features several memory and time conservation mechanisms to train multiple NN models simultaneously without impacting the accuracy of the training process. Specifically, UnifiedNN merges multiple NN models and creates a large singular unified model in order to efficiently train all models at once. We have implemented a prototype of UnifiedNN in PyTorch and we have compared its performance with relevant state-of-the-art frameworks. Our experimental results demonstrate that UnifiedNN can reduce memory consumption by up to 53% and training time by up to 81% when compared with vanilla PyTorch without impacting the model training and testing accuracy. Finally, our results indicate that UnifiedNN can reduce memory consumption by up to 52% and training time by up to 41% when compared to state-of-the-art frameworks when training multiple models concurrently.

While Large Language Models (LLM) have created a massive technological impact in the past decade, allowing for human-enabled applications, they can produce output that contains stereotypes and biases, especially when using low-resource languages. This can be of great ethical concern when dealing with sensitive topics such as religion. As a means toward making LLMS more fair, we explore bias from a religious perspective in Bengali, focusing specifically on two main religious dialects: Hindu and Muslim-majority dialects. Here, we perform different experiments and audit showing the comparative analysis of different sentences using three commonly used LLMs: ChatGPT, Gemini, and Microsoft Copilot, pertaining to the Hindu and Muslim dialects of specific words and showcasing which ones catch the social biases and which do not. Furthermore, we analyze our findings and relate them to potential reasons and evaluation perspectives, considering their global impact with over 300 million speakers worldwide. With this work, we hope to establish the rigor for creating more fairness in LLMs, as these are widely used as creative writing agents., Comment: 10 Pages, 4 Figures. Accepted to the 1st Human-centered Evaluation and Auditing of Language Models Workshop at CHI 2024 (Workshop website: https://heal-workshop.github.io/#:~:text=Exploring%20Bengali%20Religious%20Dialect%20Biases%20in%20Large%20Language%20Models%20with%20Evaluation%20Perspectives)

In this paper, the statistical properties of Newton s method algorithm output in a specific case have been studied. The relative frequency density of this sample converges to a well-defined function, prompting us to explore its distribution. Through rigorous mathematical proof, we demonstrate that the probability density function follows a Cauchy distribution. Additionally, a new method to generate a uniform distribution is proposed. To further confirm our findings, we employed statistical tests, including the Kolmogorov-Smirnov test and Anderson-Darling test, which showed high p-values. Furthermore, we show that the distribution of the distance between two successive outputs can be obtained through a transformation method applied to the Cauchy distribution.

We investigate how the Next-Two-Higgs Doublet Model extension (N2HDM) should look if we are to address the naturalness problem using dimensional regularization. In such a model, new Higgs states are predicted, namely: three CP-even $h_{1,2,3}$, one CP-odd $A$, and a pair of charged Higgs boson $H^\pm$. Our calculations of the overall quadratic divergences have been performed with full consistency with the latest data from the Large Hadron Collider (LHC) concerning the observed 125 GeV Higgs boson, alongside precision electroweak data tests and lower mass limits on charged Higgs boson. It is shown that the quadratically divergent quantum corrections $\delta_i$ (i=1,2,3) for the three CP-even Higgs bosons are controllably small, though hidden fine-tuning might still be required. This reveals a significant impact on the model parameter space, Higgs spectrum mass, and notably the singlet-doublet admixture., Comment: 27 pages, 7 figures, 9 tables

Transformers have established themselves as the leading neural network model in natural language processing and are increasingly foundational in various domains. In vision, the MLP-Mixer model has demonstrated competitive performance, suggesting that attention mechanisms might not be indispensable. Inspired by this, recent research has explored replacing attention modules with other mechanisms, including those described by MetaFormers. However, the theoretical framework for these models remains underdeveloped. This paper proposes a novel perspective by integrating Krotov's hierarchical associative memory with MetaFormers, enabling a comprehensive representation of the entire Transformer block, encompassing token-/channel-mixing modules, layer normalization, and skip connections, as a single Hopfield network. This approach yields a parallelized MLP-Mixer derived from a three-layer Hopfield network, which naturally incorporates symmetric token-/channel-mixing modules and layer normalization. Empirical studies reveal that symmetric interaction matrices in the model hinder performance in image recognition tasks. Introducing symmetry-breaking effects transitions the performance of the symmetric parallelized MLP-Mixer to that of the vanilla MLP-Mixer. This indicates that during standard training, weight matrices of the vanilla MLP-Mixer spontaneously acquire a symmetry-breaking configuration, enhancing their effectiveness. These findings offer insights into the intrinsic properties of Transformers and MLP-Mixers and their theoretical underpinnings, providing a robust framework for future model design and optimization., Comment: 18 pages

Training a proprietary Neural Network (NN) model with a proprietary dataset on the cloud comes at the risk of exposing the model architecture and the dataset to the cloud service provider. To tackle this problem, in this paper, we present an NN obfuscation framework, called Amalgam, to train NN models in a privacy-preserving manner in existing cloud-based environments. Amalgam achieves that by augmenting NN models and the datasets to be used for training with well-calibrated noise to "hide" both the original model architectures and training datasets from the cloud. After training, Amalgam extracts the original models from the augmented models and returns them to users. Our evaluation results with different computer vision and natural language processing models and datasets demonstrate that Amalgam: (i) introduces modest overheads into the training process without impacting its correctness, and (ii) does not affect the model's accuracy. The prototype implementation is available at: https://github.com/SifatTaj/amalgam

We determine the complex geometries dual to the semi-classical saddles in three-dimensional gravity with positive or negative cosmological constant. We examine the semi-classical saddles in Liouville field theory and interpret them in terms of gravity theory. For this, we describe the gravity theory by Chern Simons theory and classify the possible saddles based on homotopy group argument. We further realize the semi-classical saddles using the mini-superspace model of quantum gravity and explicitly determine the integral contour. In the case of positive cosmological constant, we recovered the geometry used for no-boundary proposal of Hartle and Hawking. In the case of negative cosmological constant, the geometry can be identified with Euclidean anti-de Sitter space attached with imaginary radius spheres. The geometry should be unphysical and several arguments on this issue are provided. Partial results were already presented in our earlier letter, and more detailed derivations and explanations on the results are given along with additional results. In particular, we reproduce the classical Liouville action from the Chern-Simons formulation of dual gravity theory., Comment: 47 pages, 6 figures

Knowledge Graphs (KGs) have proven essential in information processing and reasoning applications because they link related entities and give context-rich information, supporting efficient information retrieval and knowledge discovery; presenting information flow in a very effective manner. Despite being widely used globally, Bangla is relatively underrepresented in KGs due to a lack of comprehensive datasets, encoders, NER (named entity recognition) models, POS (part-of-speech) taggers, and lemmatizers, hindering efficient information processing and reasoning applications in the language. Addressing the KG scarcity in Bengali, we propose BanglaAutoKG, a pioneering framework that is able to automatically construct Bengali KGs from any Bangla text. We utilize multilingual LLMs to understand various languages and correlate entities and relations universally. By employing a translation dictionary to identify English equivalents and extracting word features from pre-trained BERT models, we construct the foundational KG. To reduce noise and align word embeddings with our goal, we employ graph-based polynomial filters. Lastly, we implement a GNN-based semantic filter, which elevates contextual understanding and trims unnecessary edges, culminating in the formation of the definitive KG. Empirical findings and case studies demonstrate the universal effectiveness of our model, capable of autonomously constructing semantically enriched KGs from any text., Comment: 7 pages, 3 figures. Accepted to LREC-COLING 2024. Read in ACL Anthology: https://aclanthology.org/2024.lrec-main.189/

Examining Drug-Drug Interactions (DDIs) is a pivotal element in the process of drug development. DDIs occur when one drug's properties are affected by the inclusion of other drugs. Detecting favorable DDIs has the potential to pave the way for creating and advancing innovative medications applicable in practical settings. However, existing DDI prediction models continue to face challenges related to generalization in extreme cases, robust feature extraction, and real-life application possibilities. We aim to address these challenges by leveraging the effectiveness of context-aware deep graph learning by introducing a novel framework named CADGL. Based on a customized variational graph autoencoder (VGAE), we capture critical structural and physio-chemical information using two context preprocessors for feature extraction from two different perspectives: local neighborhood and molecular context, in a heterogeneous graphical structure. Our customized VGAE consists of a graph encoder, a latent information encoder, and an MLP decoder. CADGL surpasses other state-of-the-art DDI prediction models, excelling in predicting clinically valuable novel DDIs, supported by rigorous case studies., Comment: 8 Pages, 4 Figures; In review

Recently, convolutional neural networks (CNNs) with large size kernels have attracted much attention in the computer vision field, following the success of the Vision Transformers. Large kernel CNNs have been reported to perform well in downstream vision tasks as well as in classification performance. The reason for the high-performance of large kernel CNNs in downstream tasks has been attributed to the large effective receptive field (ERF) produced by large size kernels, but this view has not been fully tested. We therefore revisit the performance of large kernel CNNs in downstream task, focusing on the weakly supervised object localization (WSOL) task. WSOL, a difficult downstream task that is not fully supervised, provides a new angle to explore the capabilities of the large kernel CNNs. Our study compares the modern large kernel CNNs ConvNeXt, RepLKNet, and SLaK to test the validity of the naive expectation that ERF size is important for improving downstream task performance. Our analysis of the factors contributing to high performance provides a different perspective, in which the main factor is feature map improvement. Furthermore, we find that modern CNNs are robust to the CAM problems of local regions of objects being activated, which has long been discussed in WSOL. CAM is the most classic WSOL method, but because of the above-mentioned problems, it is often used as a baseline method for comparison. However, experiments on the CUB-200-2011 dataset show that simply combining a large kernel CNN, CAM, and simple data augmentation methods can achieve performance (90.99% MaxBoxAcc) comparable to the latest WSOL method, which is CNN-based and requires special training or complex post-processing. The code is available at https://github.com/snskysk/CAM-Back-Again., Comment: Accepted by CVPR2024. Code: https://github.com/snskysk/CAM-Back-Again

We find out the complex geometries corresponding to the semi-classical saddles of threedimensional quantum gravity by making use of the known results of dual conformal field theory (CFT), which is effectively given by Liouville field theory. We examine both the cases with positive and negative cosmological constants. We determine the set of semi-classical saddles to choose from the homotopy argument in the Chern-Simons formulation combined with CFT results and provide strong supports from the mini-superspace approach to the quantum gravity. For the case of positive cosmological constant, partial results were already obtained in our previous works, and they are consistent with the current ones. For the case of negative cosmological constant, we identify the geometry corresponding a semi-classical saddle with three-dimensional Euclidean anti-de Sitter space dressed with imaginary radius three-dimensional spheres. The geometry is generically unphysical, but we argue that the fact itself does not lead to any problems., Comment: 7 pages, 1 figure

Complex chemical structures, like drugs, are usually defined by SMILES strings as a sequence of molecules and bonds. These SMILES strings are used in different complex machine learning-based drug-related research and representation works. Escaping from complex representation, in this work, we pose a single question: What if we treat drug SMILES as conventional sentences and engage in text classification for drug classification? Our experiments affirm the possibility with very competitive scores. The study explores the notion of viewing each atom and bond as sentence components, employing basic NLP methods to categorize drug types, proving that complex problems can also be solved with simpler perspectives. The data and code are available here: https://github.com/azminewasi/Drug-Classification-NLP., Comment: 7 pages, 2 figures, 5 tables, Accepted (invited to present) to the The Second Tiny Papers Track at ICLR 2024 (https://openreview.net/forum?id=VUYCyH8fCw)

In this paper we establish existence, uniqueness, and boundedness results for an elliptic variational inequality coupled with a nonlinear ordinary differential equation. Under the general framework, we present a new application modelling the antiplane shear deformation of a static frictional adhesive contact problem. The adhesion process has been extensively studied, but it is usual to assume a priori that the intensity of adhesion is bounded by introducing truncation operators. The aim of this article is to remove this restriction. The proof is based on an iterative approximation scheme showing that the problem has a unique solution. A key ingredient is finding uniform a priori bounds for each iterate. These are obtained by adapting versions of the Moser iteration to our system of equations., Comment: 39 pages, 1 figure