Search

We investigate whether transformers can learn to track a random process when given observations of a related process and parameters of the dynamical system that relates them as context. More specifically, we consider a finite-dimensional state-space model described by the state transition matrix $F$, measurement matrices $h_1, \dots, h_N$, and the process and measurement noise covariance matrices $Q$ and $R$, respectively; these parameters, randomly sampled, are provided to the transformer along with the observations $y_1,\dots,y_N$ generated by the corresponding linear dynamical system. We argue that in such settings transformers learn to approximate the celebrated Kalman filter, and empirically verify this both for the task of estimating hidden states $\hat{x}_{N|1,2,3,...,N}$ as well as for one-step prediction of the $(N+1)^{st}$ observation, $\hat{y}_{N+1|1,2,3,...,N}$. A further study of the transformer's robustness reveals that its performance is retained even if the model's parameters are partially withheld. In particular, we demonstrate that the transformer remains accurate at the considered task even in the absence of state transition and noise covariance matrices, effectively emulating operations of the Dual-Kalman filter.

The bimodal metallicity distribution of globular clusters (GCs) in massive galaxies implies two distinct sub-populations: metal-poor and metal-rich. Using the recent data of \textit{Gaia} we highlighted three distinct dissimilarities between metal-poor and metal-rich GCs in the Milky Way (MW). Half-mass (light) radii of metal-poor GCs exhibit, on average, $\simeq 52 \pm$5 ($60 \pm$3) per cent more expansion than metal-rich ones. Furthermore, the lack of metal-poor GCs at low Galactocentric distances ($R_\mathrm{G}$) follows a triangular pattern in $R_\mathrm{G}$-[Fe/H] space, indicating that GCs with lower metallicities appear further away from the Galactic center. Metal-poor GCs are more susceptible to destruction by the tidal field in the inner part of the MW. We perform a series of \Nbody simulations of star clusters, to study the impact of the BHs' natal kicks on the long-term evolution of low- and high-metallicity GCs to explain these observational aspects. We found that the retention of BHs inside the cluster is crucial to reproducing the observed dissimilarities. The heavier and less expanded BH sub-system (BHSub) in metal-poor clusters leads to more intense few-body encounters, injecting more kinetic energy into the stellar population. Consequently, they experience larger expansion and higher evaporation rates rather than metal-rich clusters. The higher energy production within the BHSub of metal-poor GCs causes them to dissolve before a Hubble time near the Galactic center, leading to a triangular pattern in $R_\mathrm{G}$-[Fe/H] space., Comment: 11 pages, 9 figures, 1 table. Accepted for publication in MNRAS

We present an advanced approach to medical question-answering (QA) services, using fine-tuned Large Language Models (LLMs) to improve the accuracy and reliability of healthcare information. Our study focuses on optimizing models like LLaMA-2 and Mistral, which have shown great promise in delivering precise, reliable medical answers. By leveraging comprehensive datasets, we applied fine-tuning techniques such as rsDoRA+ and ReRAG. rsDoRA+ enhances model performance through a combination of decomposed model weights, varied learning rates for low-rank matrices, and rank stabilization, leading to improved efficiency. ReRAG, which integrates retrieval on demand and question rewriting, further refines the accuracy of the responses. This approach enables healthcare providers to access fast, dependable information, aiding in more efficient decision-making and fostering greater patient trust. Our work highlights the potential of fine-tuned LLMs to significantly improve the quality and accessibility of medical information services, ultimately contributing to better healthcare outcomes for all., Comment: 8 pages, 10 figures, accepted and to be published in the proceedings of 2024 IEEE International Conference on Data Mining Workshops (ICDMW)

The optimal transport (OT) problem has gained significant traction in modern machine learning for its ability to: (1) provide versatile metrics, such as Wasserstein distances and their variants, and (2) determine optimal couplings between probability measures. To reduce the computational complexity of OT solvers, methods like entropic regularization and sliced optimal transport have been proposed. The sliced OT framework improves efficiency by comparing one-dimensional projections (slices) of high-dimensional distributions. However, despite their computational efficiency, sliced-Wasserstein approaches lack a transportation plan between the input measures, limiting their use in scenarios requiring explicit coupling. In this paper, we address two key questions: Can a transportation plan be constructed between two probability measures using the sliced transport framework? If so, can this plan be used to define a metric between the measures? We propose a "lifting" operation to extend one-dimensional optimal transport plans back to the original space of the measures. By computing the expectation of these lifted plans, we derive a new transportation plan, termed expected sliced transport (EST) plans. We prove that using the EST plan to weight the sum of the individual Euclidean costs for moving from one point to another results in a valid metric between the input discrete probability measures. We demonstrate the connection between our approach and the recently proposed min-SWGG, along with illustrative numerical examples that support our theoretical findings.

We present a refined approach to biomedical question-answering (QA) services by integrating large language models (LLMs) with Multi-BERT configurations. By enhancing the ability to process and prioritize vast amounts of complex biomedical data, this system aims to support healthcare professionals in delivering better patient outcomes and informed decision-making. Through innovative use of BERT and BioBERT models, combined with a multi-layer perceptron (MLP) layer, we enable more specialized and efficient responses to the growing demands of the healthcare sector. Our approach not only addresses the challenge of overfitting by freezing one BERT model while training another but also improves the overall adaptability of QA services. The use of extensive datasets, such as BioASQ and BioMRC, demonstrates the system's ability to synthesize critical information. This work highlights how advanced language models can make a tangible difference in healthcare, providing reliable and responsive tools for professionals to manage complex information, ultimately serving the broader goal of improved care and data-driven insights., Comment: 10 pages, 12 figures, accepted and to be published in the proceedings of 2024 IEEE International Conference on Data Mining Workshops (ICDMW)

We examine a new scenario to model the outer halo globular cluster (GC) Pal 14 over its lifetime by performing a comprehensive set of direct N-body calculations. We assume Pal 14 was born in a now detached/disrupted dwarf galaxy with a strong tidal field. Pal 14 evolved there until the slope of the stellar mass function (MF) became close to the measured value which is observed to be significantly shallower than in most GCs. After about 2-3 Gyr, Pal 14 was then captured by the Milky Way (MW). Although the physical size of such a cluster is indistinguishable from a cluster that has lived its entire life in the MW, other parameters like its mass and the MF-slope, strongly depend on the time the cluster is taken from the dwarf galaxy. After being captured by the MW on a new orbit, the cluster expands and eventually reaches the appropriate mass and size of Pal 14 after 11.5 Gyr while reproducing the observed MF. These simulations thus suggest that Pal 14 may have formed in a dwarf galaxy with a post-gas-expulsion initial half-mass radius and mass of $r_h=7$ pc and $8

We present an innovative framework that integrates consumer-grade drones into bushfire management, addressing both service improvement and data privacy concerns under Australia's Privacy Act 1988. This system establishes a marketplace where bushfire management authorities, as data consumers, access critical information from drone operators, who serve as data providers. The framework employs local differential privacy to safeguard the privacy of data providers from all system entities, ensuring compliance with privacy standards. Additionally, a blockchain-based solution facilitates fair data and fee exchanges while maintaining immutable records for enhanced accountability. Validated through a proof-of-concept implementation, the framework's scalability and adaptability make it well-suited for large-scale, real-world applications in bushfire management., Comment: 8 pages, 5 figures, accepted and to be published in the proceedings of 22nd International Conference on Service-Oriented Computing (ICSOC 2024)

Arabic handwritten text recognition (HTR) is challenging, especially for historical texts, due to diverse writing styles and the intrinsic features of Arabic script. Additionally, Arabic handwriting datasets are smaller compared to English ones, making it difficult to train generalizable Arabic HTR models. To address these challenges, we propose HATFormer, a transformer-based encoder-decoder architecture that builds on a state-of-the-art English HTR model. By leveraging the transformer's attention mechanism, HATFormer captures spatial contextual information to address the intrinsic challenges of Arabic script through differentiating cursive characters, decomposing visual representations, and identifying diacritics. Our customization to historical handwritten Arabic includes an image processor for effective ViT information preprocessing, a text tokenizer for compact Arabic text representation, and a training pipeline that accounts for a limited amount of historic Arabic handwriting data. HATFormer achieves a character error rate (CER) of 8.6% on the largest public historical handwritten Arabic dataset, with a 51% improvement over the best baseline in the literature. HATFormer also attains a comparable CER of 4.2% on the largest private non-historical dataset. Our work demonstrates the feasibility of adapting an English HTR method to a low-resource language with complex, language-specific challenges, contributing to advancements in document digitization, information retrieval, and cultural preservation.

Two-dimensional (2D) 1T-VSe$_2$ has prompted significant interest due to the discrepancies regarding alleged ferromagnetism (FM) at room temperature, charge density wave (CDW) states and the interplay between the two. We employed a combined Diffusion Monte Carlo (DMC) and density functional theory (DFT) approach to accurately investigate the magnetic properties and response of strain of monolayer 1T-VSe$_2$. Our calculations show the delicate competition between various phases, revealing critical insights into the relationship between their energetic and structural properties. We went on to perform Classical Monte Carlo simulations informed by our DMC and DFT results, and found the magnetic transition temperature ($T_c$) of the undistorted (non-CDW) FM phase to be 228 K and the distorted (CDW) phase to be 68 K. Additionally, we studied the response of biaxial strain on the energetic stability and magnetic properties of various phases of 2D 1T-VSe$_2$ and found that small amounts of strain can enhance the $T_c$, suggesting a promising route for engineering and enhancing magnetic behavior. Finally, we synthesized 1T-VSe$_2$ and performed Raman spectroscopy measurements, which were in close agreement with our calculated results. Our work emphasizes the role of highly accurate DMC methods in advancing the understanding of monolayer 1T-VSe$_2$ and provides a robust framework for future studies of 2D magnetic materials.

We investigate the impact of primordial mass segregation on the formation and evolution of dark star clusters (DSCs). Considering a wide range of initial conditions, we conducted $N$-body simulations of globular clusters (GCs) around the Milky Way. In particular, we assume a canonical IMF for all GCs without natal kicks for supernovae remnants, namely neutron stars or black holes. Our results demonstrate that clusters with larger degrees of primordial mass segregation reach their DSC phase earlier and spend a larger fraction of their dissolution time in such a phase, compared to clusters without mass segregation. In primordially segregated clusters, the maximum Galactocentric distance that the clusters can have to enter the DSC phase is almost twice that of the clusters without primordial mass segregation. Primordially segregated clusters evolve with a higher number of stellar mass black holes, accelerating energy creation in their central regions and consequently increasing evaporation rates and cluster sizes during dark phases. The simulations reveal that aggregating heavy components at the centre doubles the time spent in the dark phase. Additionally, the study identifies potential links between simulated dark clusters and initial conditions of Milky Way globular clusters, suggesting some may transition to dark phases before dissolution. Higher primordial mass segregation coefficients amplify the average binary black hole formation rate by 2.5 times, raising higher expectations for gravitational wave emissions., Comment: 10 pages, 7 Figures, 1 Table, Accepted for publication in MNRAS

Human immune system contains white blood cells (WBC) that are good indicator of many diseases like bacterial infections, AIDS, cancer, spleen, etc. White blood cells have been sub classified into four types: monocytes, lymphocytes, eosinophils and neutrophils on the basis of their nucleus, shape and cytoplasm. Traditionally in laboratories, pathologists and hematologists analyze these blood cells through microscope and then classify them manually. This manual process takes more time and increases the chance of human error. Hence, there is a need to automate this process. In this paper, first we have used different CNN pre-train models such as ResNet-50, InceptionV3, VGG16 and MobileNetV2 to automatically classify the white blood cells. These pre-train models are applied on Kaggle dataset of microscopic images. Although we achieved reasonable accuracy ranging between 92 to 95%, still there is need to enhance the performance. Hence, inspired by these architectures, a framework has been proposed to automatically categorize the four kinds of white blood cells with increased accuracy. The aim is to develop a convolution neural network (CNN) based classification system with decent generalization ability. The proposed CNN model has been tested on white blood cells images from Kaggle and LISC datasets. Accuracy achieved is 99.57% and 98.67% for both datasets respectively. Our proposed convolutional neural network-based model provides competitive performance as compared to previous results reported in literature.

Trapped-ion hardware based on the Magnetic Gradient Induced Coupling (MAGIC) scheme is emerging as a promising platform for quantum computing. Nevertheless, in this (as in any other) quantum-computing platform, many technical questions still have to be resolved before large-scale and error-tolerant applications are possible. In this work, we present a thorough discussion of the contribution of higher-order terms to the MAGIC setup, which can occur due to anharmonicities in the external potential of the ion crystal (e.g., through Coulomb repulsion) or through curvature of the applied magnetic field. These terms take the form of three-spin couplings as well as diverse terms that couple spins to phonons. We find that most of these are negligible in realistic situations, with only two contributions that need careful attention. First, there are parasitic longitudinal fields whose strength increases with chain length, but which can easily be compensated by a microwave detuning. Second, anharmonicities of the Coulomb interaction can lead to well-known two-to-one conversions of phonon excitations, which can be avoided if the phonons are ground-state cooled. Our detailed analysis constitutes an important contribution on the way of making magnetic-gradient trapped-ion quantum technology fit for large-scale applications, and it may inspire new ways to purposefully design interaction terms., Comment: 14 pages, 4 figures

We introduce jina-embeddings-v3, a novel text embedding model with 570 million parameters, achieves state-of-the-art performance on multilingual data and long-context retrieval tasks, supporting context lengths of up to 8192 tokens. The model includes a set of task-specific Low-Rank Adaptation (LoRA) adapters to generate high-quality embeddings for query-document retrieval, clustering, classification, and text matching. Evaluation on the MTEB benchmark shows that jina-embeddings-v3 outperforms the latest proprietary embeddings from OpenAI and Cohere on English tasks, while achieving superior performance compared to multilingual-e5-large-instruct across all multilingual tasks. With a default output dimension of 1024, users can flexibly reduce the embedding dimensions to as low as 32 without compromising performance, enabled by Matryoshka Representation Learning., Comment: 20 pages, pp11-13 references, pp14-20 appendix and experiment tables

Practical robotic adhoc networks (RANETs), a type of mobile wireless adhoc networks (WANETs) supporting the WiFi-Direct modes common in internet of things and phone devices, is proposed based on a strategy of exploiting WiFi-Direct connection modes to overcome hardware restrictions. For a certain period of time the community was enthusiastic about the endless opportunities in fair, robust, efficient, and cheap communication created by the Adhoc mode of the WiFi IEEE 802.11 independent basic service set (IBSS) configuration that required no dedicated access points. The mode was a main enabler of wireless Adhoc networks (WANETS). This communication mode unfortunately did not get into the standard network cards present in IoT and mobile phones, likely due to the high energy consumption it exacts. Rather, such devices implement WiFi-Direct which is designed for star topologies. Several attempts were made to overcame the restriction and support WANETs, but they break at least the fairness and symmetry property, thereby reducing applicability. Here we show a solution for fair RANETs and evaluate the behavior of various strategies using simulations., Comment: Presented at the FCRAR 24 conference in May 2024 (Florida Conference on Recent Advances in Robotics)

Multi-vector dense models, such as ColBERT, have proven highly effective in information retrieval. ColBERT's late interaction scoring approximates the joint query-document attention seen in cross-encoders while maintaining inference efficiency closer to traditional dense retrieval models, thanks to its bi-encoder architecture and recent optimizations in indexing and search. In this work we propose a number of incremental improvements to the ColBERT model architecture and training pipeline, using methods shown to work in the more mature single-vector embedding model training paradigm, particularly those that apply to heterogeneous multilingual data or boost efficiency with little tradeoff. Our new model, Jina-ColBERT-v2, demonstrates strong performance across a range of English and multilingual retrieval tasks., Comment: 8 pages, references at pp7,8; EMNLP workshop submission

We propose self-similar contracting groups as a platform for cryptographic schemes based on simultaneous conjugacy search problem (SCSP). The class of these groups contains extraordinary examples like Grigorchuk group, which is known to be non-linear, thus making some of existing attacks against SCSP inapplicable. The groups in this class admit a natural normal form based on the notion of a nucleus portrait, that plays a key role in our approach. While for some groups in the class the conjugacy search problem has been studied, there are many groups for which no algorithms solving it are known. Moreover, there are some self-similar groups with undecidable conjugacy problem. We discuss benefits and drawbacks of using these groups in group-based cryptography and provide computational analysis of variants of the length-based attack on SCSP for some groups in the class, including Grigorchuk group, Basilica group, and others., Comment: 37 pages, 13 figures

The global positioning system (GPS) has become an indispensable navigation method for field operations with unmanned surface vehicles (USVs) in marine environments. However, GPS may not always be available outdoors because it is vulnerable to natural interference and malicious jamming attacks. Thus, an alternative navigation system is required when the use of GPS is restricted or prohibited. To this end, we present a novel method that utilizes an Unmanned Aerial Vehicle (UAV) to assist in localizing USVs in GNSS-restricted marine environments. In our approach, the UAV flies along the shoreline at a consistent altitude, continuously tracking and detecting the USV using a deep learning-based approach on camera images. Subsequently, triangulation techniques are applied to estimate the USV's position relative to the UAV, utilizing geometric information and datalink range from the UAV. We propose adjusting the UAV's camera angle based on the pixel error between the USV and the image center throughout the localization process to enhance accuracy. Additionally, visual measurements are integrated into an Extended Kalman Filter (EKF) for robust state estimation. To validate our proposed method, we utilize a USV equipped with onboard sensors and a UAV equipped with a camera. A heterogeneous robotic interface is established to facilitate communication between the USV and UAV. We demonstrate the efficacy of our approach through a series of experiments conducted during the ``Muhammad Bin Zayed International Robotic Challenge (MBZIRC-2024)'' in real marine environments, incorporating noisy measurements and ocean disturbances. The successful outcomes indicate the potential of our method to complement GPS for USV navigation.

The research was conducted at the College of Agricultural Sciences Engineering/University of Sulaimani/ Kurdistan Region-Iraqi to investigate effects of different concentrations of IBA (0, 3000, 4000 and 5000 ppm) and soaking in water for 24 hours on propagation black mulberry (Morus nigra L.) by hardwood cuttings. In this research the parameters of rooting percentage, root number, root length, sprout bud number, shoot length and shoot diameter were measured. Effect of individual factors showed that the highest rooting percentage (15%) was achieved in cuttings soaked in water for 24 hours, as well as improving other traits. Also, the best (23.33%) rooting was found in cuttings dipped in 4000 ppm IBA. Interaction effects of the two factors showed that cuttings treated with 4000 ppm IBA and soaked in water for 24 hours gave the highest (40%) rooting, and the highest other root and shoot traits were achieved in the same interaction as well.

Randomization is a powerful tool that endows algorithms with remarkable properties. For instance, randomized algorithms excel in adversarial settings, often surpassing the worst-case performance of deterministic algorithms with large margins. Furthermore, their success probability can be amplified by simple strategies such as repetition and majority voting. In this paper, we enhance deep neural networks, in particular transformer models, with randomization. We demonstrate for the first time that randomized algorithms can be instilled in transformers through learning, in a purely data- and objective-driven manner. First, we analyze known adversarial objectives for which randomized algorithms offer a distinct advantage over deterministic ones. We then show that common optimization techniques, such as gradient descent or evolutionary strategies, can effectively learn transformer parameters that make use of the randomness provided to the model. To illustrate the broad applicability of randomization in empowering neural networks, we study three conceptual tasks: associative recall, graph coloring, and agents that explore grid worlds. In addition to demonstrating increased robustness against oblivious adversaries through learned randomization, our experiments reveal remarkable performance improvements due to the inherently random nature of the neural networks' computation and predictions.

The versatile nature of Visual Sentiment Analysis (VSA) is one reason for its rising profile. It isn't easy to efficiently manage social media data with visual information since previous research has concentrated on Sentiment Analysis (SA) of single modalities, like textual. In addition, most visual sentiment studies need to adequately classify sentiment because they are mainly focused on simply merging modal attributes without investigating their intricate relationships. This prompted the suggestion of developing a fusion of deep learning and machine learning algorithms. In this research, a deep feature-based method for multiclass classification has been used to extract deep features from modified ResNet50. Furthermore, gradient boosting algorithm has been used to classify photos containing emotional content. The approach is thoroughly evaluated on two benchmarked datasets, CrowdFlower and GAPED. Finally, cutting-edge deep learning and machine learning models were used to compare the proposed strategy. When compared to state-of-the-art approaches, the proposed method demonstrates exceptional performance on the datasets presented., Comment: 4 pages, 4 figures, 3 tables, IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) 2024

In this paper, we examine a discrete dynamical system defined by x(n+1) = Ax(n)+ w(n), where x takes values in a Hilbert space H and w is a periodic source with values in a fixed closed subspace W of H. Our goal is to identify conditions on some spatial sampling system G = {gj: j in J} of H that enable stable recovery of the unknown source term w from space-time samples {: n >=0,j in J}. We provide necessary and sufficient conditions on G = {g_j }_{j in J} to ensure stable recovery of any w in W . Additionally, we explicitly construct an operator R, dependent on G, such that R{}_n,j} = w., Comment: 12 pages

We show that, up to connected sums with integer homology $L$-spaces, bordered Floer homology detects handlebodies, as well as whether a mapping class extends over a given handlebody or compression body. Using this, we combine ideas of Casson-Long with the theory of train tracks to give an algorithm using bordered Floer homology to detect whether a mapping class extends over any compression body., Comment: 62 pages, 27 figures. V2: minor corrections to background

We present a model to describe the concentration-dependent growth of protein filaments. Our model contains two states, a low entropy/high affinity ordered state and a high entropy/low affinity disordered state. Consistent with experiments, our model shows a diffusion-limited linear growth regime at low concentration, followed by a concentration independent plateau at intermediate concentrations, and rapid disordered precipitation at the highest concentrations. We show that growth in the linear and plateau regions is the result of two processes that compete amid the rapid binding and unbinding of non-specific states. The first process is the addition of ordered molecules during the periods where the end of the filament is free of incorrectly bound molecules. The second process is the capture of defects, which occurs when consecutive ordered additions occur on top of incorrectly bound molecules. We show that a key molecular property is the probability that a diffusive collision results in a correctly bound state. Small values of this probability suppress the defect capture growth mode, resulting in a plateau in the growth rate when incorrectly bound molecules become common enough to poison ordered growth. We show that conditions that non-specifically suppress or enhance intermolecular interactions, such as the addition of depletants or osmolytes, have opposite effects on the growth rate in the linear and plateau regimes. In the linear regime stronger interactions promote growth by reducing dissolution events, but in the plateau regime stronger interactions inhibit growth by stabilizing incorrectly bound molecules., Comment: 20 pages, 7 figures

Effective flood hazard management requires evaluating and predicting flash flood susceptibility. Convolutional neural networks (CNNs) are commonly used for this task but face issues like gradient explosion and overfitting. This study explores the use of an attention mechanism, specifically the convolutional block attention module (CBAM), to enhance CNN models for flash flood susceptibility in the ungauged Rheraya watershed, a flood prone region. We used ResNet18, DenseNet121, and Xception as backbone architectures, integrating CBAM at different locations. Our dataset included 16 conditioning factors and 522 flash flood inventory points. Performance was evaluated using accuracy, precision, recall, F1-score, and the area under the curve (AUC) of the receiver operating characteristic (ROC). Results showed that CBAM significantly improved model performance, with DenseNet121 incorporating CBAM in each convolutional block achieving the best results (accuracy = 0.95, AUC = 0.98). Distance to river and drainage density were identified as key factors. These findings demonstrate the effectiveness of the attention mechanism in improving flash flood susceptibility modeling and offer valuable insights for disaster management.

Refractive metasurfaces (RMTSs) offer a promising solution to improve energy efficiency of wireless systems. To address the limitations of single-layer RMTS, stacked intelligent metasurfaces (SIMs), which form the desired precoder and combiner in the wave domain, have been proposed. However, previous analyses overlooked hardware non-idealities that significantly affect SIM performance. In this paper, we study the achievable sum-rate of SIM antennas in an uplink scenario, accounting for hardware constraints. We propose a system model that includes noise and hardware effects, formulate a non-convex sum-rate optimization problem, and solve it using gradient ascent and interior point methods. We compare SIMs and digital phased arrays (DPAs) under Rayleigh fading and 3GPP channels with two conditions: equal number of RF chains and equal physical aperture size. Our results show SIMs outperform DPAs under equal number of RF chains but underperform DPAs with equal aperture size.

Dynamic Metasurface Antennas (DMAs) have emerged as promising candidates for basestation deployment in the next generation of wireless communications. While overlooking the practical and hardware limitations of DMA, previous studies have highlighted DMAs' potential to deliver high data rates while maintaining low power consumption. In this paper, we address this oversight by analyzing the impact of practical hardware limitations such as antenna efficiency, power consumed in required components, processing limitations, etc. Specifically, we investigate DMA-assisted wireless communications in the uplink and propose a model which accounts for these hardware limitations. To do so, we propose a concise model to characterize the power consumption of a DMA. For a fair assessment, we propose a wave-domain combiner, based on holography theory, to maximize the achievable sum rate of DMA-assisted antennas. We compare the achievable sum rate and energy efficiency of DMA antennas with that of a partially connected hybrid phased array. Our findings reveal the true potential of DMAs when accounting for the limitations of both designs.

Many tasks can be composed from a few independent components. This gives rise to a combinatorial explosion of possible tasks, only some of which might be encountered during training. Under what circumstances can transformers compositionally generalize from a subset of tasks to all possible combinations of tasks that share similar components? Here we study a modular multitask setting that allows us to precisely control compositional structure in the data generation process. We present evidence that transformers learning in-context struggle to generalize compositionally on this task despite being in principle expressive enough to do so. Compositional generalization becomes possible only when introducing a bottleneck that enforces an explicit separation between task inference and task execution., Comment: ICML 2024 workshop on Next Generation of Sequence Modeling Architectures

Quantum devices need precise control to achieve their full capability. In this work, we address the problem of controlling closed quantum systems, tackling two main issues. First, in practice the control signals are usually subject to unknown classical distortions that could arise from the device fabrication, material properties and/or instruments generating those signals. Second, in most cases modeling the system is very difficult or not even viable due to uncertainties in the relations between some variables and inaccessibility to some measurements inside the system. In this paper, we introduce a general model-free control approach based on deep reinforcement learning (DRL), that can work for any closed quantum system. We train a deep neural network (NN), using the REINFORCE policy gradient algorithm to control the state probability distribution of a closed quantum system as it evolves, and drive it to different target distributions. We present a novel controller architecture that comprises multiple NNs. This enables accommodating as many different target state distributions as desired, without increasing the complexity of the NN or its training process. The used DRL algorithm works whether the control problem can be modeled as a Markov decision process (MDP) or a partially observed MDP. Our method is valid whether the control signals are discrete- or continuous-valued. We verified our method through numerical simulations based on a photonic waveguide array chip. We trained a controller to generate sequences of different target output distributions of the chip with fidelity higher than 99%, where the controller showed superior performance in canceling the classical signal distortions.

The frequency-dependent optical spectrum is pivotal for a broad range of applications, from material characterization to optoelectronics and energy harvesting. Data-driven surrogate models, trained on density functional theory (DFT) data, have effectively alleviated the scalability limitations of DFT while preserving its chemical accuracy, expediting material discovery. However, prevailing machine learning (ML) efforts often focus on scalar properties such as the band gap, overlooking the complexities of optical spectra. In this work, we employ deep graph neural networks (GNNs) to predict the frequency-dependent complex-valued dielectric function across the infrared, visible, and ultraviolet spectra directly from crystal structures. We explore multiple architectures for multi-output spectral representation of the dielectric function and utilize various multi-fidelity learning strategies, such as transfer learning and fidelity embedding, to address the challenges associated with the scarcity of high-fidelity DFT data. Additionally, we model key solar cell absorption efficiency metrics, demonstrating that learning these parameters is enhanced when integrated through a learning bias within the learning of the frequency-dependent absorption coefficient. This study demonstrates that leveraging multi-output and multi-fidelity ML techniques enables accurate predictions of optical spectra from crystal structures, providing a versatile tool for rapidly screening materials for optoelectronics, optical sensing, and solar energy applications across an extensive frequency spectrum.

Orthogonal delay-Doppler (DD) division multiplexing (ODDM) has been recently proposed as a promising multicarrier modulation scheme to tackle Doppler spread in high-mobility environments. Accurate channel estimation is of paramount importance to guarantee reliable communication for the ODDM, especially when the delays and Dopplers of the propagation paths are off-grid. In this paper, we propose a novel grid refinement and adjustment-based sparse Bayesian inference (GRASBI) scheme for DD domain channel estimation. The GRASBI involves first formulating the channel estimation problem as a sparse signal recovery through the introduction of a virtual DD grid. Then, an iterative process is proposed that involves (i) sparse Bayesian learning to estimate the channel parameters and (ii) a novel grid refinement and adjustment process to adjust the virtual grid points. The grid adjustment in GRASBI relies on the maximum likelihood principle to attain the adjustment and utilizes refined grids that have much higher resolution than the virtual grid. Moreover, a low-complexity grid refinement and adjustment-based channel estimation scheme is proposed, that can provides a good tradeoff between the estimation accuracy and the complexity. Finally, numerical results are provided to demonstrate the accuracy and efficiency of the proposed channel estimation schemes.

In the domain of spatial crowdsourcing drone services, which includes tasks like delivery, surveillance, and data collection, secure communication is paramount. The Public Key Infrastructure (PKI) ensures this by providing a system for digital certificates that authenticate the identities of entities involved, securing data and command transmissions between drones and their operators. However, the centralized trust model of traditional PKI, dependent on Certificate Authorities (CAs), presents a vulnerability due to its single point of failure, risking security breaches. To counteract this, the paper presents D2XChain, a blockchain-based PKI framework designed for the Internet of Drone Things (IoDT). By decentralizing the CA infrastructure, D2XChain eliminates this single point of failure, thereby enhancing the security and reliability of drone communications. Fully compatible with the X.509 standard, it integrates seamlessly with existing PKI systems, supporting all key operations such as certificate registration, validation, verification, and revocation in a distributed manner. This innovative approach not only strengthens the defense of drone services against various security threats but also showcases its practical application through deployment on a private Ethereum testbed, representing a significant advancement in addressing the unique security challenges of drone-based services and ensuring their trustworthy operation in critical tasks., Comment: 11 pages, 9 figures, accepted and to be published in the proceedings of IEEE International Conference on Web Services (ICWS 2024)

We introduce a privacy-preserving framework for integrating consumer-grade drones into bushfire management. This system creates a marketplace where bushfire management authorities obtain essential data from drone operators. Key features include local differential privacy to protect data providers and a blockchain-based solution ensuring fair data exchanges and accountability. The framework is validated through a proof-of-concept implementation, demonstrating its scalability and potential for various large-scale data collection scenarios. This approach addresses privacy concerns and compliance with regulations like Australia's Privacy Act 1988, offering a practical solution for enhancing bushfire detection and management through crowdsourced drone services., Comment: 3 pages, 2 figures, accepted and to be published in the proceedings of IEEE International Conference on Web Services (ICWS 2024)

This study introduces the Distributed Drone Reputation Management (DDRM) framework, designed to fortify trust and authenticity within the Internet of Drone Things (IoDT) ecosystem. As drones increasingly play a pivotal role across diverse sectors, integrating crowdsourced drone services within the IoDT has emerged as a vital avenue for democratizing access to these services. A critical challenge, however, lies in ensuring the authenticity and reliability of drone service reviews. Leveraging the Ethereum blockchain, DDRM addresses this challenge by instituting a verifiable and transparent review mechanism. The framework innovates with a dual-token system, comprising the Service Review Authorization Token (SRAT) for facilitating review authorization and the Drone Reputation Enhancement Token (DRET) for rewarding and recognizing drones demonstrating consistent reliability. Comprehensive analysis within this paper showcases DDRM's resilience against various reputation frauds and underscores its operational effectiveness, particularly in enhancing the efficiency and reliability of drone services., Comment: 11 pages, 1 figure, accepted and to be published in the proceedings of IEEE International Conference on Web Services (ICWS 2024)

Recent deep reinforcement learning (DRL) methods in finance show promising outcomes. However, there is limited research examining the behavior of these DRL algorithms. This paper aims to investigate their tendencies towards holding or trading financial assets as well as purchase diversity. By analyzing their trading behaviors, we provide insights into the decision-making processes of DRL models in finance applications. Our findings reveal that each DRL algorithm exhibits unique trading patterns and strategies, with A2C emerging as the top performer in terms of cumulative rewards. While PPO and SAC engage in significant trades with a limited number of stocks, DDPG and TD3 adopt a more balanced approach. Furthermore, SAC and PPO tend to hold positions for shorter durations, whereas DDPG, A2C, and TD3 display a propensity to remain stationary for extended periods.

The study was conducted to determine the best irrigation frequency for rooting hardwood cuttings of some pomegranate genotypes that are cultivated in Halabja province, Kurdistan Region, Iraq. The hardwood cuttings were collected from 11 genotypes, which were 'Salakhani Trsh' (G1), 'Salakhani Mekhosh' (G2), 'Amriki' (G3), 'Twekl Sury Trsh' (G4), 'Twekl Astury Naw Spy' (G5), 'Hanara Sherina' (G6), 'Kawa Hanary Sherin' (G7), 'Kawa Hanary Trsh' (G8), 'Malesay Twekl Asture' (G9), 'Malesay Twekl Tank' (G10), and 'Sura Hanary Trsh' (G11). The genotypes were subjected to irrigation applications by 1-day, 2-day, 7-day, or 10-day frequencies. Among pomegranates, G11, G6, and G7 produced 95, 90, and 83% rooting percentages, which were significantly higher than the rest of other genotypes. The lowest rooting percentages (28, 36, 38, and 40%) were found in G1, G5, G3, and G10, respectively. The effect of irrigation frequencies on the genotypes confirmed that a 7-day frequency was the best irrigation frequency to achieve the maximum rooting percentages (93, 86, 80, 73, 53, and 40%) in G6, G9, G2, G4, G3, and G1, respectively. In contrast, the minimum rooting percentage (20%) was recorded in G3 with a 1-day frequency and in G1 with 10-day frequency. In this study, it was found that the cuttings of G11, G6, and G7 had the best ability to form roots, and irrigation with a 7-day frequency was the best for the cuttings of all the 11 pomegranate genotypes investigated.

In this paper, we define the non-commuting graph associated to a Lie algebra L and obtain some basic graph properties such as connectivity, diameter, girth, Hamiltonian and Eulerian. Moreover, planarity, outer planarity and isomorphism between two such graphs are also discussed in the paper., Comment: 11 pages

We present the Manuscripts of Handwritten Arabic~(Muharaf) dataset, which is a machine learning dataset consisting of more than 1,600 historic handwritten page images transcribed by experts in archival Arabic. Each document image is accompanied by spatial polygonal coordinates of its text lines as well as basic page elements. This dataset was compiled to advance the state of the art in handwritten text recognition (HTR), not only for Arabic manuscripts but also for cursive text in general. The Muharaf dataset includes diverse handwriting styles and a wide range of document types, including personal letters, diaries, notes, poems, church records, and legal correspondences. In this paper, we describe the data acquisition pipeline, notable dataset features, and statistics. We also provide a preliminary baseline result achieved by training convolutional neural networks using this data.

Transformers can under some circumstances generalize to novel problem instances whose constituent parts might have been encountered during training but whose compositions have not. What mechanisms underlie this ability for compositional generalization? By reformulating multi-head attention as a hypernetwork, we reveal that a composable, low-dimensional latent code specifies key-query specific operations. We find empirically that this latent code is predictive of the subtasks the network performs on unseen task compositions revealing that latent codes acquired during training are reused to solve unseen problem instances. To further examine the hypothesis that the intrinsic hypernetwork of multi-head attention supports compositional generalization, we ablate whether making the hypernetwork generated linear value network nonlinear strengthens compositionality. We find that this modification improves compositional generalization on abstract reasoning tasks. In particular, we introduce a symbolic version of the Raven Progressive Matrices human intelligence test which gives us precise control over the problem compositions encountered during training and evaluation. We demonstrate on this task how scaling model size and data enables compositional generalization in transformers and gives rise to a functionally structured latent space., Comment: Code available at https://github.com/smonsays/hypernetwork-attention

We investigate the coexistence of orthogonal time-frequency space (OTFS) modulation with current fourth- and fifth-generation (4G/5G) communication systems that primarily use orthogonal frequency-division multiplexing (OFDM) waveforms. We first derive the input-output-relation of OTFS in the considered coexisting system. In this derivation, we consider (i) the inclusion of multiple cyclic prefixes (CPs) with unequal lengths to the OTFS signal and (ii) edge carrier unloading (ECU), to account for the impacts of CP length, frame structure, and subcarrier arrangement described in 3GPP standards for 4G/5G systems. Our analysis reveals that the inclusion of multiple CPs to the OTFS signal and ECU lead to the channel response exhibiting spreading effects/leakage along the Doppler and delay dimensions, respectively. Consequently, the effective sampled delay-Doppler (DD) domain channel model for OTFS in coexisting systems may exhibit reduced sparsity. We also show that the effective DD domain channel coefficients for OTFS in coexisting systems are influenced by the unequal lengths of CPs. Subsequently, we propose an interference cancellation-based channel estimation (CE) technique for OTFS in coexisting systems. Through numerical results, we validate our analysis, highlight the importance of not ignoring the unequal lengths of CPs during signal detection, and show the significance of the proposed CE technique., Comment: This paper has been submitted for publication in an IEEE Journal. arXiv admin note: text overlap with arXiv:2311.06850

In recent years, machine learning (ML) techniques have created numerous opportunities for intelligent mobile networks and have accelerated the automation of network operations. However, complex network tasks may involve variables and considerations even beyond the capacity of traditional ML algorithms. On the other hand, large language models (LLMs) have recently emerged, demonstrating near-human-level performance in cognitive tasks across various fields. However, they remain prone to hallucinations and often lack common sense in basic tasks. Therefore, they are regarded as assistive tools for humans. In this work, we propose the concept of "generative AI-in-the-loop" and utilize the semantic understanding, context awareness, and reasoning abilities of LLMs to assist humans in handling complex or unforeseen situations in mobile communication networks. We believe that combining LLMs and ML models allows both to leverage their respective capabilities and achieve better results than either model alone. To support this idea, we begin by analyzing the capabilities of LLMs and compare them with traditional ML algorithms. We then explore potential LLM-based applications in line with the requirements of next-generation networks. We further examine the integration of ML and LLMs, discussing how they can be used together in mobile networks. Unlike existing studies, our research emphasizes the fusion of LLMs with traditional ML-driven next-generation networks and serves as a comprehensive refinement of existing surveys. Finally, we provide a case study to enhance ML-based network intrusion detection with synthesized data generated by LLMs. Our case study further demonstrates the advantages of our proposed idea.

In this article, we explore the feedback stabilization of a viscous Burgers equation around a non-constant steady state using localized interior controls and then develop error estimates for the stabilized system using finite element method. The system is not only feedback stabilizable but exhibits an exponential decay $-\omega<0$ for any $\omega>0$. The derivation of a stabilizing control in feedback form is achieved by solving a suitable algebraic Riccati equation posed for the linearized system. In the second part of the article, we utilize a conforming finite element method to discretize the continuous system, resulting in a finite-dimensional discrete system. This approximated system is also proven to be feedback stabilizable (uniformly) with exponential decay $-\omega+\epsilon$ for any $\epsilon>0$. The feedback control for this discrete system is obtained by solving a discrete algebraic Riccati equation. To validate the effectiveness of our approach, we provide error estimates for both the stabilized solutions and the stabilizing feedback controls. Numerical implementations are carried out to support and validate our theoretical results.

Contrastive Language-Image Pretraining (CLIP) is widely used to train models to align images and texts in a common embedding space by mapping them to fixed-sized vectors. These models are key to multimodal information retrieval and related tasks. However, CLIP models generally underperform in text-only tasks compared to specialized text models. This creates inefficiencies for information retrieval systems that keep separate embeddings and models for text-only and multimodal tasks. We propose a novel, multi-task contrastive training method to address this issue, which we use to train the jina-clip-v1 model to achieve the state-of-the-art performance on both text-image and text-text retrieval tasks., Comment: 4 pages, MFM-EAI@ICML2024

The NP-hard scheduling problem P||C_max encompasses a set of tasks with known execution time which must be mapped to a set of identical machines such that the overall completion time is minimized. In this work, we improve existing techniques for optimal P||C_max scheduling with a combination of new theoretical insights and careful practical engineering. Most importantly, we derive techniques to prune vast portions of the search space of branch-and-bound (BnB) approaches. We also propose improved upper and lower bounding techniques which can be combined with any approach to P||C_max. Moreover, we present new benchmarks for P||C_max, based on diverse application data, which can shed light on aspects which prior synthetic instances fail to capture. In an extensive evaluation, we observe that our pruning techniques reduce the number of explored nodes by 90$\times$ and running times by 12$\times$. Compared to a state-of-the-art ILP-based approach, our approach is preferable for short running time limits and for instances with large makespans.