Search

Reinforcement Learning (RL) is a branch of Artificial Intelligence (AI) which focuses on training agents to make decisions by interacting with their environment to maximize cumulative rewards. An overview of RL is provided in this paper, which discusses its core concepts, methodologies, recent trends, and resources for learning. We provide a detailed explanation of key components of RL such as states, actions, policies, and reward signals so that the reader can build a foundational understanding. The paper also provides examples of various RL algorithms, including model-free and model-based methods. In addition, RL algorithms are introduced and resources for learning and implementing them are provided, such as books, courses, and online communities. This paper demystifies a comprehensive yet simple introduction for beginners by offering a structured and clear pathway for acquiring and implementing real-time techniques.

Recent advancements in learned image compression (LIC) methods have demonstrated superior performance over traditional hand-crafted codecs. These learning-based methods often employ convolutional neural networks (CNNs) or Transformer-based architectures. However, these nonlinear approaches frequently overlook the frequency characteristics of images, which limits their compression efficiency. To address this issue, we propose a novel Transformer-based image compression method that enhances the transformation stage by considering frequency components within the feature map. Our method integrates a novel Hybrid Spatial-Channel Attention Transformer Block (HSCATB), where a spatial-based branch independently handles high and low frequencies at the attention layer, and a Channel-aware Self-Attention (CaSA) module captures information across channels, significantly improving compression performance. Additionally, we introduce a Mixed Local-Global Feed Forward Network (MLGFFN) within the Transformer block to enhance the extraction of diverse and rich information, which is crucial for effective compression. These innovations collectively improve the transformation's ability to project data into a more decorrelated latent space, thereby boosting overall compression efficiency. Experimental results demonstrate that our framework surpasses state-of-the-art LIC methods in rate-distortion performance.

In this paper, we explore the problem of utilizing Integrated Access and Backhaul (IAB) technology in Non-Terrestrial Networks (NTN), with a particular focus on aerial access networks. We consider an Uncrewed Aerial Vehicle (UAV)-based wireless network comprised of two layers of UAVs: (a) a lower layer consisting a number of flying users and a UAV Base Station (BS) that provides coverage for terrestrial users and, (b) an upper layer designated to provide both wireless access for flying users and backhaul connectivity for UAV BS. By adopting IAB technology, the backhaul and access links collaboratively share their resources, enabling aerial backhauling and the utilization of the same infrastructure and frequency resources for access links. A sum-rate maximization problem is formulated by considering aerial backhaul constraints to optimally allocate the frequency spectrum between aerial and terrestrial networks. We decompose the resulting non-convex optimization problem into two sub-problems of beamforming and spectrum allocation and then propose efficient solutions for each. Numerical results in different scenarios yield insightful findings about the effectiveness of using the IAB technique in aerial networks., Comment: Accepted for publication in IEEE Open Journal of the Communications Society

Additive noise models (ANMs) are an important setting studied in causal inference. Most of the existing works on ANMs assume causal sufficiency, i.e., there are no unobserved confounders. This paper focuses on confounded ANMs, where a set of treatment variables and a target variable are affected by an unobserved confounder that follows a multivariate Gaussian distribution. We introduce a novel approach for estimating the average causal effects (ACEs) of any subset of the treatment variables on the outcome and demonstrate that a small set of interventional distributions is sufficient to estimate all of them. In addition, we propose a randomized algorithm that further reduces the number of required interventions to poly-logarithmic in the number of nodes. Finally, we demonstrate that these interventions are also sufficient to recover the causal structure between the observed variables. This establishes that a poly-logarithmic number of interventions is sufficient to infer the causal effects of any subset of treatments on the outcome in confounded ANMs with high probability, even when the causal structure between treatments is unknown. The simulation results indicate that our method can accurately estimate all ACEs in the finite-sample regime. We also demonstrate the practical significance of our algorithm by evaluating it on semi-synthetic data.

With the advances in artificial intelligence (AI), data-driven algorithms are becoming increasingly popular in the medical domain. However, due to the nonlinear and complex behavior of many of these algorithms, decision-making by such algorithms is not trustworthy for clinicians and is considered a black-box process. Hence, the scientific community has introduced explainable artificial intelligence (XAI) to remedy the problem. This systematic scoping review investigates the application of XAI in breast cancer detection and risk prediction. We conducted a comprehensive search on Scopus, IEEE Explore, PubMed, and Google Scholar (first 50 citations) using a systematic search strategy. The search spanned from January 2017 to July 2023, focusing on peer-reviewed studies implementing XAI methods in breast cancer datasets. Thirty studies met our inclusion criteria and were included in the analysis. The results revealed that SHapley Additive exPlanations (SHAP) is the top model-agnostic XAI technique in breast cancer research in terms of usage, explaining the model prediction results, diagnosis and classification of biomarkers, and prognosis and survival analysis. Additionally, the SHAP model primarily explained tree-based ensemble machine learning models. The most common reason is that SHAP is model agnostic, which makes it both popular and useful for explaining any model prediction. Additionally, it is relatively easy to implement effectively and completely suits performant models, such as tree-based models. Explainable AI improves the transparency, interpretability, fairness, and trustworthiness of AI-enabled health systems and medical devices and, ultimately, the quality of care and outcomes., Comment: 22 Pages, 6 Figures, 13 Tables

Factored Markov Decision Processes (fMDPs) are a class of Markov Decision Processes (MDPs) in which the states (and actions) can be factored into a set of state (and action) variables. The state space, action space and reward function of a fMDP can be encoded compactly using a factored representation. In this paper, we consider the setting where we have a set of potential sensors to select for the fMDP (at design-time), where each sensor measures a certain state variable and has a selection cost. We formulate the problem of selecting an optimal set of sensors for fMDPs (subject to certain budget constraints) to maximize the expected infinite-horizon discounted return provided by the optimal control policy. We show the fundamental result that it is NP-hard to approximate this optimization problem to within any non-trivial factor. We then study the dual problem of budgeted actuator selection (at design-time) to maximize the expected return under the optimal policy. Again, we show that it is NP-hard to approximate this optimization problem to within any non-trivial factor. Furthermore, with explicit examples, we show the failure of greedy algorithms for both the sensor and actuator selection problems and provide insights into the factors that cause these problems to be challenging. Despite the inapproximability results, through extensive simulations, we show that the greedy algorithm may provide near-optimal performance for actuator and sensor selection in many real-world and randomly generated fMDP instances., Comment: 22 pages, 5 figures

Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates numerous ground truth errors that obscure the true capabilities of LLMs. For example, we find that 57% of the analysed questions in the Virology subset contain errors. To address this issue, we introduce a comprehensive framework for identifying dataset errors using a novel error taxonomy. Then, we create MMLU-Redux, which is a subset of 3,000 manually re-annotated questions across 30 MMLU subjects. Using MMLU-Redux, we demonstrate significant discrepancies with the model performance metrics that were originally reported. Our results strongly advocate for revising MMLU's error-ridden questions to enhance its future utility and reliability as a benchmark. Therefore, we open up MMLU-Redux for additional annotation https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux.

Causal discovery aims to uncover cause-and-effect relationships encoded in causal graphs by leveraging observational, interventional data, or their combination. The majority of existing causal discovery methods are developed assuming infinite interventional data. We focus on data interventional efficiency and formalize causal discovery from the perspective of online learning, inspired by pure exploration in bandit problems. A graph separating system, consisting of interventions that cut every edge of the graph at least once, is sufficient for learning causal graphs when infinite interventional data is available, even in the worst case. We propose a track-and-stop causal discovery algorithm that adaptively selects interventions from the graph separating system via allocation matching and learns the causal graph based on sampling history. Given any desired confidence value, the algorithm determines a termination condition and runs until it is met. We analyze the algorithm to establish a problem-dependent upper bound on the expected number of required interventional samples. Our proposed algorithm outperforms existing methods in simulations across various randomly generated causal graphs. It achieves higher accuracy, measured by the structural hamming distance (SHD) between the learned causal graph and the ground truth, with significantly fewer samples., Comment: To appear in Proceedings of ICML 24

We consider the problem of selecting an optimal subset of information sources for a hypothesis testing/classification task where the goal is to identify the true state of the world from a finite set of hypotheses, based on finite observation samples from the sources. In order to characterize the learning performance, we propose a misclassification penalty framework, which enables nonuniform treatment of different misclassification errors. In a centralized Bayesian learning setting, we study two variants of the subset selection problem: (i) selecting a minimum cost information set to ensure that the maximum penalty of misclassifying the true hypothesis is below a desired bound and (ii) selecting an optimal information set under a limited budget to minimize the maximum penalty of misclassifying the true hypothesis. Under certain assumptions, we prove that the objective (or constraints) of these combinatorial optimization problems are weak (or approximate) submodular, and establish high-probability performance guarantees for greedy algorithms. Further, we propose an alternate metric for information set selection which is based on the total penalty of misclassification. We prove that this metric is submodular and establish near-optimal guarantees for the greedy algorithms for both the information set selection problems. Finally, we present numerical simulations to validate our theoretical results over several randomly generated instances., Comment: 21 pages, 4 figures

This systematic literature review paper explores the use of extended reality {(XR)} technology for smart built environments and particularly for smart lighting systems design. Smart lighting is a novel concept that has emerged over a decade now and is being used and tested in commercial and industrial built environments. We used PRISMA methodology to review 270 research papers published from 1968 to 2023. Following a discussion of historical advances and key modeling techniques, a description of lighting simulation in the context of extended reality and smart built environment is given, followed by a discussion of the current trends and challenges.

The brain's attention system is a complex and adaptive network of brain regions that enables individuals to interact effectively with their surroundings and perform complex tasks. This system involves the coordination of various brain regions, including the prefrontal cortex and the parietal lobes, to process and prioritize sensory information, manage tasks, and maintain focus. In this study, we investigate the intricate mechanisms underpinning the brain's attention system, followed by an exploration within the context of augmented reality (AR) settings. AR emerges as a viable technological intervention to address the multifaceted challenges faced by individuals with Attention Deficit Hyperactivity Disorder (ADHD). Given that the primary characteristics of ADHD include difficulties related to inattention, hyperactivity, and impulsivity, AR offers tailor-made solutions specifically designed to mitigate these challenges and enhance cognitive functioning. On the other hand, if these ADHD-related issues are not adequately addressed, it could lead to a worsening of their condition in AR. This underscores the importance of employing effective interventions such as AR to support individuals with ADHD in managing their symptoms. We examine the attentional mechanisms within AR environments and the sensory processing dynamics prevalent among the ADHD population. Our objective is to comprehensively address the attentional needs of this population in AR settings and offer a framework for designing cognitively accessible AR applications., Comment: 2024 HCI International Conference

The integration of brain-computer interfaces (BCIs) into the realm of smart wheelchair (SW) technology signifies a notable leap forward in enhancing the mobility and autonomy of individuals with physical disabilities. BCIs are a technology that enables direct communication between the brain and external devices. While BCIs systems offer remarkable opportunities for enhancing human-computer interaction and providing mobility solutions for individuals with disabilities, they also raise significant concerns regarding security, safety, and privacy that have not been thoroughly addressed by researchers on a large scale. Our research aims to enhance wheelchair control for individuals with physical disabilities by leveraging electroencephalography (EEG) signals for BCIs. We introduce a non-invasive BCI system that utilizes a neuro-signal acquisition headset to capture EEG signals. These signals are obtained from specific brain activities that individuals have been trained to produce, allowing for precise control of the wheelchair. EEG-based BCIs are instrumental in capturing the brain's electrical activity and translating these signals into actionable commands. The primary objective of our study is to demonstrate the system's capability to interpret EEG signals and decode specific thought patterns or mental commands issued by the user. By doing so, it aims to convert these into accurate control commands for the wheelchair. This process includes the recognition of navigational intentions, such as moving forward, backward, or executing turns, specifically tailored for wheelchair operation. Through this innovative approach, we aim to create a seamless interface between the user's cognitive intentions and the wheelchair's movements, enhancing autonomy and mobility for individuals with physical disabilities., Comment: 2024 HCI International Conference

We introduce Constellation, a dataset of 13K images suitable for research on detection of objects in dense urban streetscapes observed from high-elevation cameras, collected for a variety of temporal conditions. The dataset addresses the need for curated data to explore problems in small object detection exemplified by the limited pixel footprint of pedestrians observed tens of meters from above. It enables the testing of object detection models for variations in lighting, building shadows, weather, and scene dynamics. We evaluate contemporary object detection architectures on the dataset, observing that state-of-the-art methods have lower performance in detecting small pedestrians compared to vehicles, corresponding to a 10% difference in average precision (AP). Using structurally similar datasets for pretraining the models results in an increase of 1.8% mean AP (mAP). We further find that incorporating domain-specific data augmentations helps improve model performance. Using pseudo-labeled data, obtained from inference outcomes of the best-performing models, improves the performance of the models. Finally, comparing the models trained using the data collected in two different time intervals, we find a performance drift in models due to the changes in intersection conditions over time. The best-performing model achieves a pedestrian AP of 92.0% with 11.5 ms inference time on NVIDIA A100 GPUs, and an mAP of 95.4%.

Recent advancements in deep learning techniques have spurred considerable interest in their application to hyperspectral imagery processing. This paper provides a comprehensive review of the latest developments in this field, focusing on methodologies, challenges, and emerging trends. Deep learning architectures such as Convolutional Neural Networks (CNNs), Autoencoders, Deep Belief Networks (DBNs), Generative Adversarial Networks (GANs), and Recurrent Neural Networks (RNNs) are examined for their suitability in processing hyperspectral data. Key challenges, including limited training data and computational constraints, are identified, along with strategies such as data augmentation and noise reduction using GANs. The paper discusses the efficacy of different network architectures, highlighting the advantages of lightweight CNN models and 1D CNNs for onboard processing. Moreover, the potential of hardware accelerators, particularly Field Programmable Gate Arrays (FPGAs), for enhancing processing efficiency is explored. The review concludes with insights into ongoing research trends, including the integration of deep learning techniques into Earth observation missions such as the CHIME mission, and emphasizes the need for further exploration and refinement of deep learning methodologies to address the evolving demands of hyperspectral image processing., Comment: Hyperspectral, Deep Learning, Neural Networks, Image Processing, Classification, Segmentation, Hardware Accelerators, CHIME Mission

Iron line spectroscopy has been one of the leading methods not only for measuring the spins of accreting black holes but also for testing fundamental physics. Basing on such a method, we present an analysis of a dataset observed simultaneously by NuSTAR and NICER for the black hole binary candidate MAXI J1803-298, which shows prominent relativistic reflection features. Various relxill_nk flavors are utilized to test the Kerr black hole hypothesis. The results obtained from our analysis provide stringent constraints on Johannsen deformation parameter $\alpha_{13}$ with the highest precise to date, namely $\alpha_{13}=0.023^{+0.071}_{-0.038}$ from relxillD_nk and $\alpha_{13}=0.006^{+0.045}_{-0.022}$ from relxillion_nk respectively in 3-$\sigma$ credible lever, where Johannsen metric reduces to Kerr metric when $\alpha_{13}$ vanishes. Furthermore, we investigate the best model-fit results using Akaike Information Criterion and assess its systematic uncertainties., Comment: Accepted by ApJ

The surge in demand for efficient radio resource management has necessitated the development of sophisticated yet compact neural network architectures. In this paper, we introduce a novel approach to Graph Neural Networks (GNNs) tailored for radio resource management by presenting a new architecture: the Low Rank Message Passing Graph Neural Network (LR-MPGNN). The cornerstone of LR-MPGNN is the implementation of a low-rank approximation technique that substitutes the conventional linear layers with their low-rank counterparts. This innovative design significantly reduces the model size and the number of parameters. We evaluate the performance of the proposed LR-MPGNN model based on several key metrics: model size, number of parameters, weighted sum rate of the communication system, and the distribution of eigenvalues of weight matrices. Our extensive evaluations demonstrate that the LR-MPGNN model achieves a sixtyfold decrease in model size, and the number of model parameters can be reduced by up to 98%. Performance-wise, the LR-MPGNN demonstrates robustness with a marginal 2% reduction in the best-case scenario in the normalized weighted sum rate compared to the original MPGNN model. Additionally, the distribution of eigenvalues of the weight matrices in the LR-MPGNN model is more uniform and spans a wider range, suggesting a strategic redistribution of weights., Comment: Accepted as a full paper by the tinyML Research Symposium 2024

Wireless spectrum regulation is a complex and demanding process due to the rapid pace of technological progress, increasing demand for spectrum, and a multitude of stakeholders with potentially conflicting interests, alongside significant economic implications. To navigate this, regulators must engage effectively with all parties, keep pace with global technology trends, conduct technical evaluations, issue licenses in a timely manner, and comply with various legal and policy frameworks. In light of these challenges, this paper demonstrates example applications of Large Language Models (LLMs) to expedite spectrum regulatory processes. We explore various roles that LLMs can play in this context while identifying some of the challenges to address. The paper also offers practical case studies and insights, with appropriate experiments, highlighting the transformative potential of LLMs in spectrum management.

Ride-pooling, to gain momentum, needs to be attractive for all the parties involved. This includes also drivers, who are naturally reluctant to serve pooled rides. This can be controlled by the platform's pricing strategy, which can stimulate drivers to serve pooled rides. Here, we propose an agent-based framework, where drivers serve rides that maximise their utility. We simulate a series of scenarios in Delft and compare three strategies. Our results show that drivers, when they maximize their profits, earn more than in both the solo-rides and only-pooled rides scenarios. This shows that serving pooled rides can be beneficial as well for drivers, yet typically not all pooled rides are attractive for drivers. The proposed framework may be further applied to propose discriminative pricing in which the full potential of ride-pooling is exploited, with benefits for the platform, travellers, and (which is novel here) to the drivers., Comment: 16 pages

Federated Learning (FL) is a practical approach to train deep learning models collaboratively across user-end devices, protecting user privacy by retaining raw data on-device. In FL, participating user-end devices are highly fragmented in terms of hardware and software configurations. Such fragmentation introduces a new type of data heterogeneity in FL, namely \textit{system-induced data heterogeneity}, as each device generates distinct data depending on its hardware and software configurations. In this paper, we first characterize the impact of system-induced data heterogeneity on FL model performance. We collect a dataset using heterogeneous devices with variations across vendors and performance tiers. By using this dataset, we demonstrate that \textit{system-induced data heterogeneity} negatively impacts accuracy, and deteriorates fairness and domain generalization problems in FL. To address these challenges, we propose HeteroSwitch, which adaptively adopts generalization techniques (i.e., ISP transformation and SWAD) depending on the level of bias caused by varying HW and SW configurations. In our evaluation with a realistic FL dataset (FLAIR), HeteroSwitch reduces the variance of averaged precision by 6.3\% across device types.

A supersolid is a quantum entangled state of matter that combines features of both superfluids and solids. Despite predictions of its analog in quantum magnets, the experimental realization was lacking until recent claims in triangular-lattice compounds. Here, we report the magnetic phase diagram and neutron scattering for a spin-1/2 triangular-lattice antiferromagnet, K$_2$Co(SeO$_3$)$_2$. In zero field, neutron spectroscopy reveals the gradual development of a $\sqrt3 \times sqrt3$ magnetic order associated with $Z_3$ symmetry breaking for temperatures 5 K < T < 15 K. Below 5 K, the emergence of a Goldstone mode from low-energy continuum scattering suggests that the system enters a supersolid phase characterized by the breaking of both $Z_3$ and spin rotational U(1) symmetry. In c-axis-oriented magnetic fields 1.1 T < B < 21 T, a prominent 1/3 magnetization plateau phase emerges, accompanied by a distinct high-field supersolid phase (18 T < B < 21 T). From the coherent spin wave excitations in the 1/3 magnetized plateau phase, we infer the spin Hamiltonian, which features nearest neighbor interactions with $J_z$ = 2.98(2) meV and $J_{\rm perp}$ = 0.21(3) meV. Our work demonstrates that K$_2$Co(SeO$_3$)$_2$ is a spectacular example of a spin-1/2 triangular-lattice quantum Ising antiferromagnet, documents its magnetic phase diagram highlighting two supersolid phases, and provides spectroscopic evidence of zero-field supersolidity., Comment: 13 pages, 5 figures

We investigate how small the Randi\'c index of a graph can be in terms of its matching number, and prove several results. We give best-possible linear bounds for graphs of small excess and for subcubic graphs; in the former case the size of excess we permit is qualitatively the best possible. We show that a linear bound holds for any sparse hereditary graph class (such as planar graphs). In general, however, we show that it can be much smaller than linear. We determine the asymptotic growth rate of the minimum Randi\'c index for graphs with a near perfect matching, and conjecture that the same bounds hold for all graphs., Comment: 11 pages, 2 figures. Added the observation that the assumptions in Corollary 6 are necessary, and other minor remarks

We proceed to investigate the solutions of generalized supergravity equations (GSE) in three dimensions. Our candidate is the metric of BTZ black hole. It is shown that only the cases with $J=M=0$ and $J=0,~ M\neq 0$ of the BTZ metric satisfy the GSE. In the former, we find a family of solutions including the field strength $H_{_{r \varphi t}}=2r/l$, the cosmological constant $\Lambda=-1/l^2$, one-form $Z_{\mu}$ and a vector field which is obtained to be a linear combination of the directions of the time translation and rotational symmetries. In the latter, the solutions possess the same field strength as before, while the cosmological constant $\Lambda$, one-form $Z_{\mu}$ and vector field $I$ will be different from the previous case. Finally, we show that the charged black string solution found by Horne and Horowitz, which is Abelian T-dual to the the BTZ black hole solution, can be considered as a solution for the GSE., Comment: 6 pages, footnote 3 and Ref. 22 have been added

The global agrochemical market is highly consolidated, with large multinational companies accounting for a major share of the market. Thus, even for a single agrochemical product, its global supply chain typically involves numerous paths connecting the raw material sources to the final customers. Besides structural complexity, agrochemical supply chains are also subject to seasonality and other unique uncertainties, thereby posing a need for risk management tools and strategies. In this study, we model and optimize an agrochemcial supply chain by developing and solving a stochastic mixed-integer quadratic constrained program (MIQCP). We model and control the demand uncertainty in this scenario-based MIQCP using variance. We also apply perspective reformulation techniques to convert the MIQCP to a mixed-integer linear program (MILP). Computational experiment results from an illustrative example show that, successively introducing perspective cuts to the reformulated MILP not only leads to a tight approximation of the original MIQCP model, but is also more computationally efficient than directly solving the MIQCP., Comment: Presented at the 2023 IISE Annual Conference and Expo