Search

When predicting the next token in a sequence, vanilla transformers compute attention over all previous tokens, resulting in quadratic scaling of compute with sequence length. State-space models compress the entire sequence of tokens into a fixed-dimensional representation to improve efficiency, while other architectures achieve sub-quadratic complexity via low-rank projections or sparse attention patterns over the sequence. In this paper, we introduce Attamba, a novel architecture that uses state-space models to compress chunks of tokens and applies attention on these compressed key-value representations. We find that replacing key and value projections in a transformer with SSMs can improve model quality and enable flexible token chunking, resulting in 24% improved perplexity with transformer of similar KV-Cache and attention footprint, and ~4 times smaller KV-Cache and Attention FLOPs for 5% perplexity trade-off. Attamba can perform attention on chunked-sequences of variable length, enabling a smooth transition between quadratic and linear scaling, offering adaptable efficiency gains.

The Jansen-Rit model of a cortical column in the cerebral cortex is widely used to simulate spontaneous brain activity (EEG) and event-related potentials. It couples a pyramidal cell population with two interneuron populations, of which one is fast and excitatory and the other slow and inhibitory. Our paper studies the transition between alpha and delta oscillations produced by the model. Delta oscillations are slower than alpha oscillations and have a more complex relaxation-type time profile. In the context of neuronal population activation dynamics, a small threshold means that neurons begin to activate with small input or stimulus, indicating high sensitivity to incoming signals. A steep slope signifies that activation increases sharply as input crosses the threshold. Accordingly in the model the excitatory activation thresholds are small and the slopes are steep. Hence, a singular limit replacing the excitatory activation function with all-or-nothing switches, eg. a Heaviside function, is appropriate. In this limit we identify the transition between alpha and delta oscillations as a discontinuity-induced grazing bifurcation. At the grazing the minimum of the pyramidal-cell output equals the threshold for switching off the excitatory interneuron population, leading to a collapse in excitatory feedback.

Novel reconfigurable intelligent surface (RIS) architectures, known as beyond diagonal RISs (BD-RISs), have been proposed to enhance reflection efficiency and expand RIS capabilities. However, their passive nature, non-diagonal reflection matrix, and the large number of coupled reflecting elements complicate the channel state information (CSI) estimation process. The challenge further escalates in scenarios with fast-varying channels. In this paper, we address this challenge by proposing novel joint channel estimation and prediction strategies with low overhead and high accuracy for two different RIS architectures in a BD-RIS-assisted multiple-input multiple-output system under correlated fast-fading environments with channel aging. The channel estimation procedure utilizes the Tucker2 decomposition with bilinear alternative least squares, which is exploited to decompose the cascade channels of the BD-RIS-assisted system into effective channels of reduced dimension. The channel prediction framework is based on a convolutional neural network combined with an autoregressive predictor. The estimated/predicted CSI is then utilized to optimize the RIS phase shifts aiming at the maximization of the downlink sum rate. Insightful simulation results demonstrate that our proposed approach is robust to channel aging, and exhibits a high estimation accuracy. Moreover, our scheme can deliver a high average downlink sum rate, outperforming other state-of-the-art channel estimation methods. The results also reveal a remarkable reduction in pilot overhead of up to 98\% compared to baseline schemes, all imposing low computational complexity., Comment: arXiv admin note: text overlap with arXiv:2406.07387

Our future society will be increasingly digitalised, hyper-connected and globally data driven. The sixth generation (6G) and beyond 6G wireless networks are expected to bridge the digital and physical worlds by providing wireless connectivity as a service to different vertical sectors, including industries, smart cities, eHealth and autonomous transportation. Such far reaching integration will render the society increasingly reliant on wireless networks. While this has the potential to greatly enhance our quality and ease of life, any disruption to these networks would also have significant impact with overreaching consequences. Disruptions can happen due to a variety of reasons, including planned outages, failures due to the nature of wireless propagation, natural disasters, and deliberate cybersecurity attacks. Hence, 6G and beyond 6G networks should not only provide near instant and virtually unlimited connectivity, but also be resilient against internal and external disruptions. This paper proposes a resilient-by-design framework towards this end. First, we provide an overview of the disruption landscape. Thereafter, we comprehensively outline the main features of the proposed concept. Finally, we detail the four key steps of the framework, namely predict, preempt, protect and progress. A simple but illustrative preliminary simulation result is also presented to highlight the potential advantages and efficiency of the proposed approach in addressing outages., Comment: Submitted to IEEE Communications Magazine

With the rapid growth of technology, emerging IT professionals increasingly require mentorship to secure positions in the field. Recognizing this need, ReCoded has enhanced the skill set of their tech Bootcamp graduates by introducing the community platform -- "ReCoded's Mentorship Platform". To improve the user experience of the volunteer mentors at ReCoded, this thesis investigates optimizing mentors' interactions with digital mentorship platforms and provides suggestions for enhancing these interactions. Multiple third-party collaborators have powered the mentorship platform at ReCoded. This thesis examines the platform powered by StellarUp as a case study. The insights obtained may inform the UX design of any subsequent mentorship platforms considered by ReCoded. This thesis adopted a user-centric approach to solving a UX question. The study identified challenges in the mentors' user journey and their needs by engaging with users via interviews, usability tests, and eye-tracking methods. Three principal issues emerged: platform navigation, the onboarding process, and the seamless integration of external tools. Solutions were derived from desk research for onboarding, card sorting techniques for navigation, and competitive analyses for tool integration. Throughout the research, feedback from 23 participants was gathered, ensuring a holistic understanding and actionable recommendations for developing a user-friendly and efficient mentorship platform., Comment: Master's thesis

Vocabulary adaptation, which integrates new vocabulary into pre-trained language models (LMs), enables expansion to new languages and mitigates token over-fragmentation. However, existing approaches are limited by their reliance on heuristic or external embeddings. We propose VocADT, a novel method for vocabulary adaptation using adapter modules that are trained to learn the optimal linear combination of existing embeddings while keeping the model's weights fixed. VocADT offers a flexible and scalable solution without requiring external resources or language constraints. Across 11 languages-with various scripts, resource availability, and fragmentation-we demonstrate that VocADT outperforms the original Mistral model and other baselines across various multilingual tasks. We find that Latin-script languages and highly fragmented languages benefit the most from vocabulary adaptation. We further fine-tune the adapted model on the generative task of machine translation and find that vocabulary adaptation is still beneficial after fine-tuning and that VocADT is the most effective method.

We evaluate the performance of the LoRaWAN Long-Range Frequency Hopping Spread Spectrum (LR-FHSS) technique using a device-level probabilistic strategy for code rate and header replica allocation. Specifically, we investigate the effects of different header replica and code rate allocations at each end-device, guided by a probability distribution provided by the network server. As a benchmark, we compare the proposed strategy with the standardized LR-FHSS data rates DR8 and DR9. Our numerical results demonstrate that the proposed strategy consistently outperforms the DR8 and DR9 standard data rates across all considered scenarios. Notably, our findings reveal that the optimal distribution rarely includes data rate DR9, while data rate DR8 significantly contributes to the goodput and energy efficiency optimizations.

Large language models (LLMs) have achieved remarkable success across various NLP tasks, yet their focus has predominantly been on English due to English-centric pre-training and limited multilingual data. While some multilingual LLMs claim to support for hundreds of languages, models often fail to provide high-quality response for mid- and low-resource languages, leading to imbalanced performance heavily skewed in favor of high-resource languages like English and Chinese. In this paper, we prioritize quality over scaling number of languages, with a focus on multilingual machine translation task, and introduce X-ALMA, a model designed with a commitment to ensuring top-tier performance across 50 diverse languages, regardless of their resource levels. X-ALMA surpasses state-of-the-art open-source multilingual LLMs, such as Aya-101 and Aya-23, in every single translation direction on the FLORES and WMT'23 test datasets according to COMET-22. This is achieved by plug-and-play language-specific module architecture to prevent language conflicts during training and a carefully designed training regimen with novel optimization methods to maximize the translation performance. At the final stage of training regimen, our proposed Adaptive Rejection Preference Optimization (ARPO) surpasses existing preference optimization methods in translation tasks.

Crack detection, particularly from pavement images, presents a formidable challenge in the domain of computer vision due to several inherent complexities such as intensity inhomogeneity, intricate topologies, low contrast, and noisy backgrounds. Automated crack detection is crucial for maintaining the structural integrity of essential infrastructures, including buildings, pavements, and bridges. Existing lightweight methods often face challenges including computational inefficiency, complex crack patterns, and difficult backgrounds, leading to inaccurate detection and impracticality for real-world applications. To address these limitations, we propose EfficientCrackNet, a lightweight hybrid model combining Convolutional Neural Networks (CNNs) and transformers for precise crack segmentation. EfficientCrackNet integrates depthwise separable convolutions (DSC) layers and MobileViT block to capture both global and local features. The model employs an Edge Extraction Method (EEM) and for efficient crack edge detection without pretraining, and Ultra-Lightweight Subspace Attention Module (ULSAM) to enhance feature extraction. Extensive experiments on three benchmark datasets Crack500, DeepCrack, and GAPs384 demonstrate that EfficientCrackNet achieves superior performance compared to existing lightweight models, while requiring only 0.26M parameters, and 0.483 FLOPs (G). The proposed model offers an optimal balance between accuracy and computational efficiency, outperforming state-of-the-art lightweight models, and providing a robust and adaptable solution for real-world crack segmentation.

Selecting an automatic metric that best emulates human annotators is often non-trivial, because there is no clear definition of "best emulates." A meta-metric is required to compare the human judgments to the automatic metric scores, and metric rankings depend on the choice of meta-metric. We propose Soft Pairwise Accuracy (SPA), a new meta-metric that builds on Pairwise Accuracy (PA) but incorporates the statistical significance of both the human judgments and the metric scores. We show that SPA is more stable than PA with respect to changes in the number of systems/segments used for evaluation. We also show that PA can only assign a small set of distinct output values to metrics, and this results in many metrics being artificially assigned the exact same PA score. We demonstrate that SPA fixes this issue. Finally, we show that SPA is more discriminative than PA, producing more statistically significant comparisons between metrics. SPA was selected as the official system-level metric for the 2024 WMT Metrics Shared Task., Comment: Accepted at WMT 2024

In this paper, we present an improved version of the digital signature scheme proposed by Sharafi and Daghigh based on Module-LWE and Module-SIS problems. Our proposed signature scheme has a notably higher security level and smaller decoding failure probability, than the ones in the Sharaf-Daghigh scheme, at the expense of enlarging the module of the underlying basic ring.

In this paper, we intend to investigate the dynamics of the Circular Restricted Three-Body Problem. Here we assumed the primaries as the source of radiation and have variable mass. The gravitational perturbation from disk-like structure are also considered in this study. There exist five equilibrium points in this system. By considering the combined effect from disk-like structure and the mass transfer, we found that the classical collinear equilibrium points depart from x-axis. Meanwhile, this combined effect also breaks the symmetry of tringular equlibrium point positions. We noted that the quasi-equilibrium points are unstable whereas the triangular equilibrium points are stable if the mass ratio $\mu$ smaller than critical mass $\mu_c$. It shows that the stability of triangular equilibrium points depends on time., Comment: 15 pages, 5 figures, 2 tables. Accepted for publication in Romanian Astronomical Journal (RoAJ)

The high power consumption and latency-sensitive deployments of large language models (LLMs) have motivated efficiency techniques like quantization and sparsity. Contextual sparsity, where the sparsity pattern is input-dependent, is crucial in LLMs because the permanent removal of attention heads or neurons from LLMs can significantly degrade accuracy. Prior work has attempted to model contextual sparsity using neural networks trained to predict activation magnitudes, which can be used to dynamically prune structures with low predicted activation magnitude. In this paper, we look beyond magnitude-based pruning criteria to assess attention head and neuron importance in LLMs. We develop a novel predictor called ShadowLLM, which can shadow the LLM behavior and enforce better sparsity patterns, resulting in over 15% improvement in end-to-end accuracy compared to prior methods. In addition, ShadowLLM achieves up to a 20% speed-up over the state-of-the-art DejaVu framework. These enhancements are validated on Llama-2 and OPT models with up to 30 billion parameters. Our code is available at \href{https://github.com/abdelfattah-lab/shadow_llm/}{ShadowLLM}., Comment: Accepted to EMNLP 2024 (Main, Long Paper)

Reconfigurable intelligent surfaces (RISs) have emerged as a promising technology to enhance the performance of sixth-generation (6G) and beyond communication systems. The passive nature of RISs and their large number of reflecting elements pose challenges to the channel estimation process. The associated complexity further escalates when the channel coefficients are fast-varying as in scenarios with user mobility. In this paper, we propose an extended channel estimation framework for RIS-assisted multiple-input multiple-output (MIMO) systems based on a convolutional neural network (CNN) integrated with an autoregressive (AR) predictor. The implemented framework is designed for identifying the aging pattern and predicting enhanced estimates of the wireless channels in correlated fast-fading environments. Insightful simulation results demonstrate that our proposed CNN-AR approach is robust to channel aging, exhibiting a high-precision estimation accuracy. The results also show that our approach can achieve high spectral efficiency and low pilot overhead compared to traditional methods.

Recent advancements in Artificial Intelligence (AI) and machine learning have demonstrated transformative capabilities across diverse domains. This progress extends to the field of patent analysis and innovation, where AI-based tools present opportunities to streamline and enhance important tasks in the patent cycle such as classification, retrieval, and valuation prediction. This not only accelerates the efficiency of patent researchers and applicants but also opens new avenues for technological innovation and discovery. Our survey provides a comprehensive summary of recent AI tools in patent analysis from more than 40 papers from 26 venues between 2017 and 2023. Unlike existing surveys, we include methods that work for patent image and text data. Furthermore, we introduce a novel taxonomy for the categorization based on the tasks in the patent life cycle as well as the specifics of the AI methods. This interdisciplinary survey aims to serve as a resource for researchers and practitioners who are working at the intersection of AI and patent analysis as well as the patent offices that are aiming to build efficient patent systems.

In order to deal with imprecision, ambiguity, and uncertainty in data analysis, Pawlak introduced rough set theory in 1982. This paper aims to expand the scope of basic set theory developed by presenting the notions of $\delta\mathbb{P}$-upper and $\delta\mathbb{P}$-lower approximations, that are based on the notion of $\delta\mathbb{P}$-open sets, we additionally examine a few of their fundamental characteristics., Comment: 14 pages, 1 figure, 1 table

Wireless communication systems must increasingly support a multitude of machine-type communications (MTC) devices, thus calling for advanced strategies for active user detection (AUD). Recent literature has delved into AUD techniques based on compressed sensing, highlighting the critical role of signal sparsity. This study investigates the relationship between frequency diversity and signal sparsity in the AUD problem. Single-antenna users transmit multiple copies of non-orthogonal pilots across multiple frequency channels and the base station independently performs AUD in each channel using the orthogonal matching pursuit algorithm. We note that, although frequency diversity may improve the likelihood of successful reception of the signals, it may also damage the channel sparsity level, leading to important trade-offs. We show that a sparser signal significantly benefits AUD, surpassing the advantages brought by frequency diversity in scenarios with limited temporal resources and/or high numbers of receive antennas. Conversely, with longer pilots and fewer receive antennas, investing in frequency diversity becomes more impactful, resulting in a tenfold AUD performance improvement., Comment: 5 pages, 5 figures. Manuscript submitted to IEEE Wireless Communications Letters for review

The International Celestial Reference Frame (ICRF) plays an important role in astronomy and geodesy. The realization of ICRF is based on the position of thousands of quasars observed using the Very-Long Baseline Interferometry (VLBI) technique. Better quality of ICRF is achieved when the position of the quasars is stable. In this study, we aim to analyze the stability of one of the quasars in ICRF called 4C31.61 (2201+315). We performed VLBI data analysis by using Vienna VLBI and Satellite Software (VieVS) to get the position of the quasar. We also used the data of the quasar's position from the Paris Observatory Geodetic VLBI Center. We examined the stability of the quasar position by using the Allan standard deviation technique. We found that the quasar 4C31.61 (2201+315) has a stable position with the dominance of white noise across the majority of time scales., Comment: to be published in IOP Journal of Physics Conference Series (JPCS)

The synchronisation between rotating turbulent flows in periodic boxes is investigated numerically. The flows are coupled via a master-slave coupling, taking the Fourier modes with wavenumber below a given value $k_m$ as the master modes. It is found that synchronisation happens when $k_m$ exceeds a threshold value $k_c$, and $k_c$ depends strongly on the forcing scheme. In rotating Kolmogorov flows, $k_c\eta$ does not change with rotation in the range of rotation rates considered, $\eta$ being the Kolmogorov length scale. Even though the energy spectrum has a steeper slope, the value of $k_c\eta$ is the same as that found in isotropic turbulence. In flows driven by a forcing term maintaining constant energy injection rate, synchronisation becomes easier when rotation is stronger. $k_c\eta$ decreases with rotation, and it is reduced significantly for strong rotations when the slope of the energy spectrum approaches $-3$. It is shown that the conditional Lyapunov exponent for a given $k_m$ is reduced by rotation in the flows driven by the second type of forcing, but it increases mildly with rotation for the Kolmogorov flows. The local conditional Lyapunov exponents fluctuate more strongly as rotation is increased, although synchronisation occurs as long as the average conditional Lyapunov exponents are negative. We also look for the relationship between $k_c$ and the energy spectra of the Lyapunov vectors. We find that the spectra always seem to peak around $k_c$, and synchronisation fails when the energy spectra of the conditional Lyapunov vectors have a local maximum in the slaved modes.

In this proceedings contribution, we summarize recent findings concerning the presence of early- and late-time attractors in non-conformal kinetic theory. We study the effects of varying both the initial momentum-space anisotropy and initialization times using an exact solution of the 0+1D boost-invariant Boltzmann equation with a mass- and temperature-dependent relaxation time. Our findings support the existence of a longitudinal pressure attractor, but they do not support the existence of distinct attractors for the bulk viscous and shear pressures. Considering a large set of integral moments, we show that for moments with greater than one power of longitudinal momentum squared, both early- and late-time attractors are present., Comment: 4 pages, 2figures; Proceedings contribution to Quark Matter 2023, September 3-9 2023, Houston, Texas, USA

We propose the on-the-fly ensembling of a machine translation model with an LLM, prompted on the same task and input. We perform experiments on 4 language pairs (both directions) with varying data amounts. We find that a slightly weaker-at-translation LLM can improve translations of a NMT model, and ensembling with an LLM can produce better translations than ensembling two stronger MT models. We combine our method with various techniques from LLM prompting, such as in context learning and translation context.

Managing inter-cell interference is among the major challenges in a wireless network, more so when strict quality of service needs to be guaranteed such as in ultra-reliable low latency communications (URLLC) applications. This study introduces a novel intelligent interference management framework for a local 6G network that allocates resources based on interference prediction. The proposed algorithm involves an advanced signal pre-processing technique known as empirical mode decomposition followed by prediction of each decomposed component using the sequence-to-one transformer algorithm. The predicted interference power is then used to estimate future signal-to-interference plus noise ratio, and subsequently allocate resources to guarantee the high reliability required by URLLC applications. Finally, an interference cancellation scheme is explored based on the predicted interference signal with the transformer model. The proposed sequence-to-one transformer model exhibits its robustness for interference prediction. The proposed scheme is numerically evaluated against two baseline algorithms, and is found that the root mean squared error is reduced by up to 55% over a baseline scheme., Comment: Accepted for IEEE GLOBECOM 2023 Workshops, Kuala Lumpur, Malaysia

We intend to study a modified version of the planar Circular Restricted Three-Body Problem (CRTBP) by incorporating several perturbing parameters. We consider the bigger primary as an oblate spheroid and emitting radiation while the small primary has an elongated body. We also consider the perturbation from a disk-like structure encompassing this three-body system. First, we develop a mathematical model of this modified CRTBP. We have found there exist five equilibrium points in this modified CRTBP model, where three of them are collinear and the other two are non-collinear. Second, we apply our modified CRTBP model to the Sun-Haumea system by considering several values of each perturbing parameter. Through our numerical investigation, we have discovered that the incorporation of perturbing parameters has resulted in a shift in the equilibrium point positions of the Sun-Haumea system compared to their positions in the classical CRTBP. The stability of equilibrium points is investigated. We have shown that the collinear equilibrium points are unstable and the stability of non-collinear equilibrium points depends on the mass parameter $\mu$ of the system. Unlike the classical case, non-collinear equilibrium points have both a maximum and minimum limit of $\mu$ for achieving stability. We remark that the stability range of $\mu$ in non-collinear equilibrium points depends on the perturbing parameters. In context of the Sun-Haumea system, we have found that the non-collinear equilibrium points are stable., Comment: Accepted for publication in Research in Astronomy and Astrophysics

Many machine translation toolkits make use of a data preparation step wherein raw data is transformed into a tensor format that can be used directly by the trainer. This preparation step is increasingly at odds with modern research and development practices because this process produces a static, unchangeable version of the training data, making common training-time needs difficult (e.g., subword sampling), time-consuming (preprocessing with large data can take days), expensive (e.g., disk space), and cumbersome (managing experiment combinatorics). We propose an alternative approach that separates the generation of data from the consumption of that data. In this approach, there is no separate pre-processing step; data generation produces an infinite stream of permutations of the raw training data, which the trainer tensorizes and batches as it is consumed. Additionally, this data stream can be manipulated by a set of user-definable operators that provide on-the-fly modifications, such as data normalization, augmentation or filtering. We release an open-source toolkit, SOTASTREAM, that implements this approach: https://github.com/marian-nmt/sotastream. We show that it cuts training time, adds flexibility, reduces experiment management complexity, and reduces disk space, all without affecting the accuracy of the trained models.

Invariant scattering transform introduces new area of research that merges the signal processing with deep learning for computer vision. Nowadays, Deep Learning algorithms are able to solve a variety of problems in medical sector. Medical images are used to detect diseases brain cancer or tumor, Alzheimer's disease, breast cancer, Parkinson's disease and many others. During pandemic back in 2020, machine learning and deep learning has played a critical role to detect COVID-19 which included mutation analysis, prediction, diagnosis and decision making. Medical images like X-ray, MRI known as magnetic resonance imaging, CT scans are used for detecting diseases. There is another method in deep learning for medical imaging which is scattering transform. It builds useful signal representation for image classification. It is a wavelet technique; which is impactful for medical image classification problems. This research article discusses scattering transform as the efficient system for medical image analysis where it's figured by scattering the signal information implemented in a deep convolutional network. A step by step case study is manifested at this research work., Comment: 11 pages, 8 figures and 1 table

There are many complex and rich morphological subtleties in the Arabic language, which are very useful when analyzing traditional Arabic texts, especially in the historical and religious contexts, and help in understanding the meaning of the texts. Vocabulary separation means separating the word into different parts such as root and affix. In the morphological datasets, the variety of labels and the number of data samples helps to evaluate the morphological methods. In this paper, we present a benchmark data set for evaluating the methods of separating Arabic words which include about 223,690 words from the book of Sharia alIslam, which have been labeled by experts. In terms of the volume and variety of words, this dataset is superior to other existing data sets, and as far as we know, there are no Arabic Hadith Domain texts. To evaluate the dataset, we applied different methods such as Farasa, Camel, Madamira, and ALP to the dataset and we reported the annotation quality through four evaluation methods., Comment: 15 pages, 2 figures

We investigate how to use the indicatrix of an invariant metric to rescale a sequence of biholomorphic maps and to ensure the convergence of the rescaled sequence. We also use the indicatrix of the Kobayashi-Royden metric to define the maximal circularity function which indicates how the domain is close to a circular domain., Comment: 13 pages

We release MMSMR, a Massively Multi-System MultiReference dataset to enable future work on metrics and evaluation for dialog. Automatic metrics for dialogue evaluation should be robust proxies for human judgments; however, the verification of robustness is currently far from satisfactory. To quantify the robustness correlation and understand what is necessary in a test set, we create and release an 8-reference dialog dataset by extending single-reference evaluation sets and introduce this new language learning conversation dataset. We then train 1750 systems and evaluate them on our novel test set and the DailyDialog dataset. We release the novel test set, and model hyper parameters, inference outputs, and metric scores for each system on a variety of datasets.

Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astronomers will celebrate their accomplishments for the life of the mission, potentially as long as 20 years, and beyond. This report and the scientific discoveries that follow are extended thank-you notes to the 20,000 team members. The telescope is working perfectly, with much better image quality than expected. In this and accompanying papers, we give a brief history, describe the observatory, outline its objectives and current observing program, and discuss the inventions and people who made it possible. We cite detailed reports on the design and the measured performance on orbit., Comment: Accepted by PASP for the special issue on The James Webb Space Telescope Overview, 29 pages, 4 figures

Effective resource allocation is a crucial requirement to achieve the stringent performance targets of ultra-reliable low-latency communication (URLLC) services. Predicting future interference and utilizing it to design efficient interference management algorithms is one way to allocate resources for URLLC services effectively. This paper proposes an empirical mode decomposition (EMD) based hybrid prediction method to predict the interference and allocate resources for downlink based on the prediction results. EMD is used to decompose the past interference values faced by the user equipment. Long short-term memory and auto-regressive integrated moving average methods are used to predict the decomposed components. The final predicted interference value is reconstructed using individual predicted values of decomposed components. It is found that such a decomposition-based prediction method reduces the root mean squared error of the prediction by $20 - 25\%$. The proposed resource allocation algorithm utilizing the EMD-based interference prediction was found to meet near-optimal allocation of resources and correspondingly results in $2-3$ orders of magnitude lower outage compared to state-of-the-art baseline prediction algorithm-based resource allocation., Comment: Accepted for European Conference on Networks and Communications (EUCNC) and 6G Summit-2023, Gothenburg, Sweden

Current deep models provide remarkable object detection in terms of object classification and localization. However, estimating object rotation with respect to other visual objects in the visual context of an input image still lacks deep studies due to the unavailability of object datasets with rotation annotations. This paper tackles these two challenges to solve the rotation estimation of a parked bike with respect to its parking area. First, we leverage the power of 3D graphics to build a camera-agnostic well-annotated Synthetic Bike Rotation Dataset (SynthBRSet). Then, we propose an object-to-spot rotation estimator (OSRE) by extending the object detection task to further regress the bike rotations in two axes. Since our model is purely trained on synthetic data, we adopt image smoothing techniques when deploying it on real-world images. The proposed OSRE is evaluated on synthetic and real-world data providing promising results. Our data and code are available at \href{https://github.com/saghiralfasly/OSRE-Project}{https://github.com/saghiralfasly/OSRE-Project}.

The Near-Infrared Spectrograph (NIRSpec) is one of the four focal plane instruments on the James Webb Space Telescope. In this paper, we summarize the in-orbit performance of NIRSpec, as derived from data collected during its commissioning campaign and the first few months of nominal science operations. More specifically, we discuss the performance of some critical hardware components such as the two NIRSpec Hawaii-2RG (H2RG) detectors, wheel mechanisms, and the micro-shutter array. We also summarize the accuracy of the two target acquisition procedures used to accurately place science targets into the slit apertures, discuss the current status of the spectro-photometric and wavelength calibration of NIRSpec spectra, and provide the as measured sensitivity in all NIRSpec science modes. Finally, we point out a few important considerations for the preparation of NIRSpec science programs., Comment: accepted by PASP for special issue on JWST in-orbit performance

Process analytic approaches play a critical role in supporting the practice of business process management and continuous process improvement by leveraging process-related data to identify performance bottlenecks, extracting insights about reducing costs and optimizing the utilization of available resources. Process analytic techniques often have to contend with real-world settings where available logs are noisy or incomplete. In this paper we present an approach that permits process analytics techniques to deliver value in the face of noisy/incomplete event logs. Our approach leverages knowledge graphs to mitigate the effects of noise in event logs while supporting process analysts in understanding variability associated with event logs.

A modern-day society demands resilient, reliable, and smart urban infrastructure for effective and in telligent operations and deployment. However, unexpected, high-impact, and low-probability events such as earthquakes, tsunamis, tornadoes, and hurricanes make the design of such robust infrastructure more complex. As a result of such events, a power system infrastructure can be severely affected, leading to unprecedented events, such as blackouts. Nevertheless, the integration of smart grids into the existing framework of smart cities adds to their resilience. Therefore, designing a resilient and reliable power system network is an inevitable requirement of modern smart city infras tructure. With the deployment of the Internet of Things (IoT), smart cities infrastructures have taken a transformational turn towards introducing technologies that do not only provide ease and comfort to the citizens but are also feasible in terms of sustainability and dependability. This paper presents a holistic view of a resilient and sustainable smart city architecture that utilizes IoT, big data analytics, unmanned aerial vehicles, and smart grids through intelligent integration of renew able energy resources. In addition, the impact of disasters on the power system infrastructure is investigated and different types of optimization techniques that can be used to sustain the power flow in the network during disturbances are compared and analyzed. Furthermore, a comparative review analysis of different data-driven machine learning techniques for sustainable smart cities is performed along with the discussion on open research issues and challenges.

In many emerging Internet of Things (IoT) applications, the freshness of the is an important design criterion. Age of Information (AoI) quantifies the freshness of the received information or status update. This work considers a setup of deployed IoT devices in an IoT network; multiple unmanned aerial vehicles (UAVs) serve as mobile relay nodes between the sensors and the base station. We formulate an optimization problem to jointly plan the UAVs' trajectory, while minimizing the AoI of the received messages and the devices' energy consumption. The solution accounts for the UAVs' battery lifetime and flight time to recharging depots to ensure the UAVs' green operation. The complex optimization problem is efficiently solved using a deep reinforcement learning algorithm. In particular, we propose a deep Q-network, which works as a function approximation to estimate the state-action value function. The proposed scheme is quick to converge and results in a lower ergodic age and ergodic energy consumption when compared with benchmark algorithms such as greedy algorithm (GA), nearest neighbour (NN), and random-walk (RW)., Comment: in IEEE Transactions on Vehicular Technology, 2022. arXiv admin note: text overlap with arXiv:2209.09206

We compute bottomonium suppression and elliptic flow within the pNRQCD effective field theory using an open quantum systems approach. For the hydrodynamical background, we use 2+1D MUSIC second-order viscous hydrodynamics with IP-Glasma initial conditions and evolve bottom/antibottom quantum wave packets in real time in these backgrounds. We find that the impact of fluctuating initial conditions is small when compared to results obtained using smooth initial conditions. Including the effect of fluctuating initial conditions, we find that the Upsilon(1S) integrated elliptic flow is v_2[1S] = 0.005 +/- 0.002 +/- 0.001, with the first and second variations corresponding to statistical and systematic theoretical uncertainties, respectively., Comment: 7 pages, 3 figures; 2 page supplement with 3 figures

State government operations comprise a large number of transactions for different processes that must be carried out across the state. This comprises new projects, maintenance and repairs, public employee compensation, and agricultural schemes. Low-level corruption, which is sometimes difficult to trace and hinders state growth, is a big challenge for the top administration. In order to eradicate corruption and bring transparency, technology can be used in an efficient way. An important task to exterminate corruption is to keep track of all the financial transactions of an undergoing project. This research uses blockchain technology to keep track of fund management systems and assure the transparency of any financial statement. This paper proposes to use a gateway where all transaction records are updated in the system and visible to all stakeholders. We find research gaps in the literature and focus on including government funds and local currency usage. The proposed model's motive is to generate a funding model that attains two sub-goals: designing a fund management methodology in which authorized individuals can receive and withdraw allocated funds in crypto currency, and evaluating a smart contract to incorporate the money and identify transparency and tracking. The proposed model executes every feature of our system in just 8.3786ms on average., Comment: arXiv admin note: text overlap with arXiv:2203.03169 by other authors. arXiv admin note: substantial text overlap with arXiv:2203.03169 by other authors without attribution

Video-based dialog task is a challenging multimodal learning task that has received increasing attention over the past few years with state-of-the-art obtaining new performance records. This progress is largely powered by the adaptation of the more powerful transformer-based language encoders. Despite this progress, existing approaches do not effectively utilize visual features to help solve tasks. Recent studies show that state-of-the-art models are biased toward textual information rather than visual cues. In order to better leverage the available visual information, this study proposes a new framework that combines 3D-CNN network and transformer-based networks into a single visual encoder to extract more robust semantic representations from videos. The visual encoder is jointly trained end-to-end with other input modalities such as text and audio. Experiments on the AVSD task show significant improvement over baselines in both generative and retrieval tasks.

We introduce a Transformer based 6D Object Pose Estimation framework VideoPose, comprising an end-to-end attention based modelling architecture, that attends to previous frames in order to estimate accurate 6D Object Poses in videos. Our approach leverages the temporal information from a video sequence for pose refinement, along with being computationally efficient and robust. Compared to existing methods, our architecture is able to capture and reason from long-range dependencies efficiently, thus iteratively refining over video sequences. Experimental evaluation on the YCB-Video dataset shows that our approach is on par with the state-of-the-art Transformer methods, and performs significantly better relative to CNN based approaches. Further, with a speed of 33 fps, it is also more efficient and therefore applicable to a variety of applications that require real-time object pose estimation. Training code and pretrained models are available at https://github.com/ApoorvaBeedu/VideoPose, Comment: arXiv admin note: text overlap with arXiv:2111.10677

We investigate whether early and late time attractors for non-conformal kinetic theories exist by computing the time-evolution of a large set of moments of the one-particle distribution function. For this purpose we make use of a previously obtained exact solution of the 0+1D boost-invariant massive Boltzmann equation in relaxation time approximation. We extend prior attractor studies of non-conformal systems by using a realistic mass- and temperature-dependent relaxation time and explicitly computing the effect of varying both the initial momentum-space anisotropy and initialization time on the time evolution of a large set of integral moments. Our findings are consistent with prior studies, which found that there is an attractor for the scaled longitudinal pressure, but not for the shear and bulk viscous corrections separately. We further present evidence that both late- and early-time attractors exist for all moments of the one-particle distribution function that contain greater than one power of the longitudinal momentum squared., Comment: 25 pages, 12 figures; v3: fix expression for the non-conformal Navier-Stokes shear-viscous correction; small changes in figures; no changes in the exact solution or conclusions

Many emerging Internet of Things (IoT) applications rely on information collected by sensor nodes where the freshness of information is an important criterion. \textit{Age of Information} (AoI) is a metric that quantifies information timeliness, i.e., the freshness of the received information or status update. This work considers a setup of deployed sensors in an IoT network, where multiple unmanned aerial vehicles (UAVs) serve as mobile relay nodes between the sensors and the base station. We formulate an optimization problem to jointly plan the UAVs' trajectory, while minimizing the AoI of the received messages. This ensures that the received information at the base station is as fresh as possible. The complex optimization problem is efficiently solved using a deep reinforcement learning (DRL) algorithm. In particular, we propose a deep Q-network, which works as a function approximation to estimate the state-action value function. The proposed scheme is quick to converge and results in a lower AoI than the random walk scheme. Our proposed algorithm reduces the average age by approximately $25\%$ and requires down to $50\%$ less energy when compared to the baseline scheme.

The NIRSpec instrument on the James Webb Space Telescope (JWST) brings the first multi-object spectrograph (MOS) into space, enabled by a programmable Micro Shutter Array (MSA) of ~250,000 individual apertures. During the 6-month Commissioning period, the MSA performed admirably, completing ~800 reconfigurations with an average success rate of ~96% for commanding shutters open in science-like patterns. We show that 82.5% of the unvignetted shutter population is usable for science, with electrical short masking now the primary cause of inoperable apertures. In response, we propose a plan to recheck existing shorts during nominal operations, which is expected to reduce the number of affected shutters. We also present a full assessment of the Failed Open and Failed Closed shutter populations, which both show a marginal increase in line with predictions from ground testing. We suggest an amendment to the Failed Closed shutter flagging scheme to improve flexibility for MSA configuration planning. Overall, the NIRSpec MSA performed very well during Commissioning, and the MOS mode was declared ready for science operations on schedule., Comment: 15 pages, 6 figures, to appear in Proceedings of SPIE conference "Space Telescopes and Instrumentation 2022: Optical, Infrared, and Millimeter Wave"

The Near-Infrared Spectrograph (NIRSpec) is one of the four focal plane instruments on the James Webb Space Telescope which was launched on Dec. 25, 2021. We present an overview of the as-run NIRSpec commissioning campaign, with particular emphasis on the sequence of activities that led to the verification of all hardware components of NIRSpec. We also discuss the mechanical, thermal, and operational performance of NIRSpec, as well as the readiness of all NIRSpec observing modes for use in the upcoming JWST science program., Comment: 10 pages, to appear in Proceedings of SPIE conference "Space Telescopes and Instrumentation 2022: Optical, Infrared, and Millimeter Wave"

This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performance of the spacecraft, telescope, science instruments, and ground system, with an emphasis on differences from pre-launch expectations. Commissioning has made clear that JWST is fully capable of achieving the discoveries for which it was built. Moreover, almost across the board, the science performance of JWST is better than expected; in most cases, JWST will go deeper faster than expected. The telescope and instrument suite have demonstrated the sensitivity, stability, image quality, and spectral range that are necessary to transform our understanding of the cosmos through observations spanning from near-earth asteroids to the most distant galaxies., Comment: 5th version as accepted to PASP; 31 pages, 18 figures; https://iopscience.iop.org/article/10.1088/1538-3873/acb293

Future industrial applications will encompass compelling new use cases requiring stringent performance guarantees over multiple key performance indicators (KPI) such as reliability, dependability, latency, time synchronization, security, etc. Achieving such stringent and diverse service requirements necessitates the design of a special-purpose Industrial Internet of Things (IIoT) network comprising a multitude of specialized functionalities and technological enablers. This article proposes an innovative architecture for such a special-purpose 6G IIoT network incorporating seven functional building blocks categorized into: special-purpose functionalities and enabling technologies. The former consists of Wireless Environment Control, Traffic/Channel Prediction, Proactive Resource Management and End-to-End Optimization functions; whereas the latter includes Synchronization and Coordination, Machine Learning and Artificial Intelligence Algorithms, and Auxiliary Functions. The proposed architecture aims at providing a resource-efficient and holistic solution for the complex and dynamically challenging requirements imposed by future 6G industrial use cases. Selected test scenarios are provided and assessed to illustrate cross-functional collaboration and demonstrate the applicability of the proposed architecture in a wireless IIoT network., Comment: 11 pages, 5 figures

In this paper, a multi-objective optimization problem (MOOP) is proposed for maximizing the achievable finite blocklength (FBL) rate while minimizing the utilized channel blocklengths (CBLs) in a reconfigurable intelligent surface (RIS)-assisted short packet communication system. The formulated MOOP has two objective functions namely maximizing the total FBL rate with a target error probability, and minimizing the total utilized CBLs which is directly proportional to the transmission duration. The considered MOOP variables are the base station (BS) transmit power, number of CBLs, and passive beamforming at the RIS. Since the proposed non-convex problem is intractable to solve, the Tchebyshev method is invoked to transform it into a single-objective OP, then the alternating optimization (AO) technique is employed to iteratively obtain optimized parameters in three main sub-problems. The numerical results show a fundamental trade-off between maximizing the achievable rate in the FBL regime and reducing the transmission duration. Also, the applicability of RIS technology is emphasized in reducing the utilized CBLs while increasing the achievable rate significantly., Comment: Accepted in IEEE Communications Letters

Prior work on integrating text corpora with knowledge graphs (KGs) to improve Knowledge Graph Embedding (KGE) have obtained good performance for entities that co-occur in sentences in text corpora. Such sentences (textual mentions of entity-pairs) are represented as Lexicalised Dependency Paths (LDPs) between two entities. However, it is not possible to represent relations between entities that do not co-occur in a single sentence using LDPs. In this paper, we propose and evaluate several methods to address this problem, where we borrow LDPs from the entity pairs that co-occur in sentences in the corpus (i.e. with mention entity pairs) to represent entity pairs that do not co-occur in any sentence in the corpus (i.e. without mention entity pairs). We propose a supervised borrowing method, SuperBorrow, that learns to score the suitability of an LDP to represent a without-mention entity pair using pre-trained entity embeddings and contextualised LDP representations. Experimental results show that SuperBorrow improves the link prediction performance of multiple widely-used prior KGE methods such as TransE, DistMult, ComplEx and RotatE., Comment: Accepted in NAACL 2022