Search

Causal discovery with time series data remains a challenging yet increasingly important task across many scientific domains. Convergent cross mapping (CCM) and related methods have been proposed to study time series that are generated by dynamical systems, where traditional approaches like Granger causality are unreliable. However, CCM often yields inaccurate results depending upon the quality of the data. We propose the Tangent Space Causal Inference (TSCI) method for detecting causalities in dynamical systems. TSCI works by considering vector fields as explicit representations of the systems' dynamics and checks for the degree of synchronization between the learned vector fields. The TSCI approach is model-agnostic and can be used as a drop-in replacement for CCM and its generalizations. We first present a basic version of the TSCI algorithm, which is shown to be more effective than the basic CCM algorithm with very little additional computation. We additionally present augmented versions of TSCI that leverage the expressive power of latent variable models and deep learning. We validate our theory on standard systems, and we demonstrate improved causal inference performance across a number of benchmark tasks., Comment: 18 pages, 7 figures. Accepted to NeurIPS 2024

Random feature latent variable models (RFLVMs) represent the state-of-the-art in latent variable models, capable of handling non-Gaussian likelihoods and effectively uncovering patterns in high-dimensional data. However, their heavy reliance on Monte Carlo sampling results in scalability issues which makes it difficult to use these models for datasets with a massive number of observations. To scale up RFLVMs, we turn to the optimization-based variational Bayesian inference (VBI) algorithm which is known for its scalability compared to sampling-based methods. However, implementing VBI for RFLVMs poses challenges, such as the lack of explicit probability distribution functions (PDFs) for the Dirichlet process (DP) in the kernel learning component, and the incompatibility of existing VBI algorithms with RFLVMs. To address these issues, we introduce a stick-breaking construction for DP to obtain an explicit PDF and a novel VBI algorithm called ``block coordinate descent variational inference" (BCD-VI). This enables the development of a scalable version of RFLVMs, or in short, SRFLVM. Our proposed method shows scalability, computational efficiency, superior performance in generating informative latent representations and the ability of imputing missing data across various real-world datasets, outperforming state-of-the-art competitors.

Counterfactual reasoning allows us to explore hypothetical scenarios in order to explain the impacts of our decisions. However, addressing such inquires is impossible without establishing the appropriate mathematical framework. In this work, we introduce the problem of counterfactual reasoning in the context of vector autoregressive (VAR) processes. We also formulate the inference of a causal model as a joint regression task where for inference we use both data with and without interventions. After learning the model, we exploit linearity of the VAR model to make exact predictions about the effects of counterfactual interventions. Furthermore, we quantify the total causal effects of past counterfactual interventions. The source code for this project is freely available at https://github.com/KurtButler/counterfactual_interventions.

Online prediction of time series under regime switching is a widely studied problem in the literature, with many celebrated approaches. Using the non-parametric flexibility of Gaussian processes, the recently proposed INTEL algorithm provides a product of experts approach to online prediction of time series under possible regime switching, including the special case of outliers. This is achieved by adaptively combining several candidate models, each reporting their predictive distribution at time $t$. However, the INTEL algorithm uses a finite context window approximation to the predictive distribution, the computation of which scales cubically with the maximum lag, or otherwise scales quartically with exact predictive distributions. We introduce LINTEL, which uses the exact filtering distribution at time $t$ with constant-time updates, making the time complexity of the streaming algorithm optimal. We additionally note that the weighting mechanism of INTEL is better suited to a mixture of experts approach, and propose a fusion policy based on arithmetic averaging for LINTEL. We show experimentally that our proposed approach is over five times faster than INTEL under reasonable settings with better quality predictions., Comment: 8 pages, 4 figures. Accepted to the International Conference on Information Fusion 2024 (FUSION 2024)

In this paper, we consider a scenario with one UAV equipped with a ULA, which sends combined information and sensing signals to communicate with multiple GBS and, at the same time, senses potential targets placed within an interested area on the ground. We aim to jointly design the transmit beamforming with the GBS association to optimize communication performance while ensuring high sensing accuracy. We propose a predictive beamforming framework based on a dual DNN solution to solve the formulated nonconvex optimization problem. A first DNN is trained to produce the required beamforming matrix for any point of the UAV flying area in a reduced time compared to state-of-the-art beamforming optimizers. A second DNN is trained to learn the optimal mapping from the input features, power, and EIRP constraints to the GBS association decision. Finally, we provide an extensive simulation analysis to corroborate the proposed approach and show the benefits of EIRP, SINR performance and computational speed.

Practical Bayesian learning often requires (1) online inference, (2) dynamic models, and (3) ensembling over multiple different models. Recent advances have shown how to use random feature approximations to achieve scalable, online ensembling of Gaussian processes with desirable theoretical properties and fruitful applications. One key to these methods' success is the inclusion of a random walk on the model parameters, which makes models dynamic. We show that these methods can be generalized easily to any basis expansion model and that using alternative basis expansions, such as Hilbert space Gaussian processes, often results in better performance. To simplify the process of choosing a specific basis expansion, our method's generality also allows the ensembling of several entirely different models, for example, a Gaussian process and polynomial regression. Finally, we propose a novel method to ensemble static and dynamic models together., Comment: 34 pages, 14 figures. Accepted to Transactions on Machine Learning Research (TMLR)

The field of explainable artificial intelligence (XAI) attempts to develop methods that provide insight into how complicated machine learning methods make predictions. Many methods of explanation have focused on the concept of feature attribution, a decomposition of the model's prediction into individual contributions corresponding to each input feature. In this work, we explore the problem of feature attribution in the context of Gaussian process regression (GPR). We take a principled approach to defining attributions under model uncertainty, extending the existing literature. We show that although GPR is a highly flexible and non-parametric approach, we can derive interpretable, closed-form expressions for the feature attributions. When using integrated gradients as an attribution method, we show that the attributions of a GPR model also follow a Gaussian process distribution, which quantifies the uncertainty in attribution arising from uncertainty in the model. We demonstrate, both through theory and experimentation, the versatility and robustness of this approach. We also show that, when applicable, the exact expressions for GPR attributions are both more accurate and less computationally expensive than the approximations currently used in practice. The source code for this project is freely available under MIT license at https://github.com/KurtButler/2024_attributions_paper., Comment: 38 pages, 7 figures

In science and engineering, we often work with models designed for accurate prediction of variables of interest. Recognizing that these models are approximations of reality, it becomes desirable to apply multiple models to the same data and integrate their outcomes. In this paper, we operate within the Bayesian paradigm, relying on Gaussian processes as our models. These models generate predictive probability density functions (pdfs), and the objective is to integrate them systematically, employing both linear and log-linear pooling. We introduce novel approaches for log-linear pooling, determining input-dependent weights for the predictive pdfs of the Gaussian processes. The aggregation of the pdfs is realized through Monte Carlo sampling, drawing samples of weights from their posterior. The performance of these methods, as well as those based on linear pooling, is demonstrated using a synthetic dataset.

We introduce Dagma-DCE, an interpretable and model-agnostic scheme for differentiable causal discovery. Current non- or over-parametric methods in differentiable causal discovery use opaque proxies of ``independence'' to justify the inclusion or exclusion of a causal relationship. We show theoretically and empirically that these proxies may be arbitrarily different than the actual causal strength. Juxtaposed to existing differentiable causal discovery algorithms, \textsc{Dagma-DCE} uses an interpretable measure of causal strength to define weighted adjacency matrices. In a number of simulated datasets, we show our method achieves state-of-the-art level performance. We additionally show that \textsc{Dagma-DCE} allows for principled thresholding and sparsity penalties by domain-experts. The code for our method is available open-source at https://github.com/DanWaxman/DAGMA-DCE, and can easily be adapted to arbitrary differentiable models., Comment: 9 pages, 2 figures. Accepted to the IEEE Open Journal of Signal Processing

Fusing probabilistic information is a fundamental task in signal and data processing with relevance to many fields of technology and science. In this work, we investigate the fusion of multiple probability density functions (pdfs) of a continuous random variable or vector. Although the case of continuous random variables and the problem of pdf fusion frequently arise in multisensor signal processing, statistical inference, and machine learning, a universally accepted method for pdf fusion does not exist. The diversity of approaches, perspectives, and solutions related to pdf fusion motivates a unified presentation of the theory and methodology of the field. We discuss three different approaches to fusing pdfs. In the axiomatic approach, the fusion rule is defined indirectly by a set of properties (axioms). In the optimization approach, it is the result of minimizing an objective function that involves an information-theoretic divergence or a distance measure. In the supra-Bayesian approach, the fusion center interprets the pdfs to be fused as random observations. Our work is partly a survey, reviewing in a structured and coherent fashion many of the concepts and methods that have been developed in the literature. In addition, we present new results for each of the three approaches. Our original contributions include new fusion rules, axioms, and axiomatic and optimization-based characterizations; a new formulation of supra-Bayesian fusion in terms of finite-dimensional parametrizations; and a study of supra-Bayesian fusion of posterior pdfs for linear Gaussian models.

Future networks of unmanned aerial vehicles (UAVs) will be tasked to carry out ever-increasing complex operations that are time-critical and that require accurate localization performance (e.g., tracking the state of a malicious user). Since there is the need to preserve low UAV complexity while tackling the challenging goals of missions in effective ways, one key aspect is the UAV intelligence (UAV-I). The UAV's intelligence includes the UAV's capability to process information and to make decisions, e.g., to decide where to sense and whether to delegate some tasks to other network entities. In this paper, we provide an overview of possible solutions for the design of UAVs of low complexity, showing some of the needs of the UAVs for running efficient localization operations, performed either as a team or individually. Further, we focus on different network configurations, which possibly include assistance with edge computing. We also discuss open problems and future perspectives for these settings.

Applications towards 6G have brought a huge interest towards arrays with a high number of antennas and operating within the millimeter and sub-THz bandwidths for joint communication and localization. With such large arrays, the plane wave approximation is often not accurate because the system may operate in the near-field propagation region (Fresnel region) where the electromagnetic field wavefront is spherical. In this case, the curvature of arrival (CoA) is a measure of the spherical wavefront that can be used to infer the source position using only a single large array. In this paper, we study a near-field tracking problem for inferring the state (i.e., the position and velocity) of a moving source with an ad-hoc observation model that accounts for the phase profile of a large receiving array. For this tracking problem, we derive the posterior Cram\'er-Rao Lower Bound (P-CRLB) and show the effects when the source moves inside and outside the Fresnel region. We provide insights on how the loss of positioning information outside Fresnel comes from an increase of the ranging error rather than from inaccuracies of angular estimation. Then, we investigate the performance of different Bayesian tracking algorithms in the presence of model mismatches and abrupt trajectory changes. Our results demonstrate the feasibility and high accuracy for most of the tracking approaches without the need of wideband signals and of any synchronization scheme. signals and of any synchronization scheme.

In a number of data-driven applications such as detection of arrhythmia, interferometry or audio compression, observations are acquired indistinctly in the time or frequency domains: temporal observations allow us to study the spectral content of signals (e.g., audio), while frequency-domain observations are used to reconstruct temporal/spatial data (e.g., MRI). Classical approaches for spectral analysis rely either on i) a discretisation of the time and frequency domains, where the fast Fourier transform stands out as the \textit{de facto} off-the-shelf resource, or ii) stringent parametric models with closed-form spectra. However, the general literature fails to cater for missing observations and noise-corrupted data. Our aim is to address the lack of a principled treatment of data acquired indistinctly in the temporal and frequency domains in a way that is robust to missing or noisy observations, and that at the same time models uncertainty effectively. To achieve this aim, we first define a joint probabilistic model for the temporal and spectral representations of signals, to then perform a Bayesian model update in the light of observations, thus jointly reconstructing the complete (latent) time and frequency representations. The proposed model is analysed from a classical spectral analysis perspective, and its implementation is illustrated through intuitive examples. Lastly, we show that the proposed model is able to perform joint time and frequency reconstruction of real-world audio, healthcare and astronomy signals, while successfully dealing with missing data and handling uncertainty (noise) naturally against both classical and modern approaches for spectral estimation., Comment: 14 pages, 16 figures

In this paper, we consider a new framework for particle filtering under model uncertainty that operates beyond the scope of Markovian switching systems. Specifically, we develop a novel particle filtering algorithm that applies to general regime switching systems, where the model index is augmented as an unknown time-varying parameter in the system. The proposed approach does not require the use of multiple filters and can maintain a diverse set of particles for each considered model through appropriate choice of the particle filtering proposal distribution. The flexibility of the proposed approach allows for long-term dependencies between the models, which enables its use to a wider variety of real-world applications. We validate the method on a synthetic data experiment and show that it outperforms state-of-the-art multiple model particle filtering approaches that require the use of multiple filters., Comment: Accepted to EUSIPCO 2020

In this paper, we study a joint detection, mapping and navigation problem for a single unmanned aerial vehicle (UAV) equipped with a low complexity radar and flying in an unknown environment. The goal is to optimize its trajectory with the purpose of maximizing the mapping accuracy and, at the same time, to avoid areas where measurements might not be sufficiently informative from the perspective of a target detection. This problem is formulated as a Markov decision process (MDP) where the UAV is an agent that runs either a state estimator for target detection and for environment mapping, and a reinforcement learning (RL) algorithm to infer its own policy of navigation (i.e., the control law). Numerical results show the feasibility of the proposed idea, highlighting the UAV's capability of autonomously exploring areas with high probability of target detection while reconstructing the surrounding environment.

Nowadays there is a growing research interest on the possibility of enriching small flying robots with autonomous sensing and online navigation capabilities. This will enable a large number of applications spanning from remote surveillance to logistics, smarter cities and emergency aid in hazardous environments. In this context, an emerging problem is to track unauthorized small unmanned aerial vehicles (UAVs) hiding behind buildings or concealing in large UAV networks. In contrast with current solutions mainly based on static and on-ground radars, this paper proposes the idea of a dynamic radar network of UAVs for real-time and high-accuracy tracking of malicious targets. To this end, we describe a solution for real-time navigation of UAVs to track a dynamic target using heterogeneously sensed information. Such information is shared by the UAVs with their neighbors via multi-hops, allowing tracking the target by a local Bayesian estimator running at each agent. Since not all the paths are equal in terms of information gathering point-of-view, the UAVs plan their own trajectory by minimizing the posterior covariance matrix of the target state under UAV kinematic and anti-collision constraints. Our results show how a dynamic network of radars attains better localization results compared to a fixed configuration and how the on-board sensor technology impacts the accuracy in tracking a target with different radar cross sections, especially in non line-of-sight (NLOS) situations.

We investigate the performance of a class of particle filters (PFs) that can automatically tune their computational complexity by evaluating online certain predictive statistics which are invariant for a broad class of state-space models. To be specific, we propose a family of block-adaptive PFs based on the methodology of Elvira et al (2017). In this class of algorithms, the number of Monte Carlo samples (known as particles) is adjusted periodically, and we prove that the theoretical error bounds of the PF actually adapt to the updates in the number of particles. The evaluation of the predictive statistics that lies at the core of the methodology is done by generating fictitious observations, i.e., particles in the observation space. We study, both analytically and numerically, the impact of the number $K$ of these particles on the performance of the algorithm. In particular, we prove that if the predictive statistics with $K$ fictitious observations converged exactly, then the particle approximation of the filtering distribution would match the first $K$ elements in a series of moments of the true filter. This result can be understood as a converse to some convergence theorems for PFs. From this analysis, we deduce an alternative predictive statistic that can be computed (for some models) without sampling any fictitious observations at all. Finally, we conduct an extensive simulation study that illustrates the theoretical results and provides further insights into the complexity, performance and behavior of the new class of algorithms.

In this paper, we propose a joint indoor localization and navigation algorithm to enable a swarm of unmanned aerial vehicles (UAVs) to deploy in a specific spatial formation in indoor environments. In the envisioned scenario, we consider a static user acting as a central unit whose main task is to acquire all the UAV measurements carrying position-dependent information and to estimate the UAV positions when there is no existing infrastructure for positioning. Subsequently, the user exploits the estimated positions as inputs for the navigation control with the aim of deploying the UAVs in a desired formation in space (formation shaping). The user plans the trajectory of each UAV in real time, guaranteeing a safe navigation in the presence of obstacles. The proposed algorithm guides the UAVs to their desired final locations with good accuracy.

In this paper, we propose a distributed solution to the navigation of a population of unmanned aerial vehicles (UAVs) to best localize a static source. The network is considered heterogeneous with UAVs equipped with received signal strength (RSS) sensors from which it is possible to estimate the distance from the source and/or the direction of arrival through ad-hoc rotations. This diversity in gathering and processing RSS measurements mitigates the loss of localization accuracy due to the adoption of low-complexity sensors. The UAVs plan their trajectories on-the-fly and in a distributed fashion. The collected data are disseminated through the network via multi-hops, therefore being subject to latency. Since not all the paths are equal in terms of information gathering rewards, the motion planning is formulated as a minimization of the uncertainty of the source position under UAV kinematic and anti-collision constraints and performed by 3D non-linear programming. The proposed analysis takes into account non-line-of-sight (NLOS) channel conditions as well as measurement age caused by the latency constraints in communication., Comment: 5 pages, 3 figures, conference

In the incoming years, the low-aerial space will be crowded by unmanned aerial vehicles (UAVs), which will be providing different services. In this expected context, an emerging problem is to detect and track unauthorized or malicious mini/micro UAVs. In contrast to current solutions mainly based on fixed terrestrial radars, this article will put forth the idea of a dynamic radar network (DRN) composed of UAVs able to smartly adapt their formation-navigation control to best track malicious UAVs in real-time, with high accuracy, and in a distributed fashion. To this end, some technological solutions and the main methods for target detection and tracking will be described. Further, an optimized navigation scheme will be developed according to an information-seeking approach. Some examples of simulation results and future directions of work will be finally presented highlighting the advantages of dynamic and reconfigurable networks over static ones.

In this paper, we address the estimation of a time-varying spatial field of received signal strength (RSS) by relying on measurements from randomly placed and not very accurate sensors. We employ a radio propagation model where the path loss exponent and the transmitted power are unknown with Gaussian priors whose hyper-parameters are estimated by applying the empirical Bayes method. We consider the locations of the sensors to be imperfectly known, which entails that they represent another source of error in the model. The propagation model includes shadowing which is considered to be a zero-mean Gaussian process where the correlation of attenuation between two spatial points is quantified by an exponential function of the distance between the points. The location of the transmitter is also unknown and estimated from the data with a weighted centroid approach. We propose to estimate time-varying RSS fields by a recursive Bayesian method and crowdsourcing. The method is based on Gaussian processes (GP), and it produces the joint distribution of the spatial field. Further, it summarizes all the acquired information by keeping the size of the needed memory bounded. We also present the Bayesian Cram\'er-Rao bound (BCRB) of the estimated parameters. Finally, we illustrate the performance of our method with experimental results on synthetic data sets., Comment: This manuscript has been submitted to IEEE Transactions on Signal Processing on May 11, 2018; revised on August 9, 2018 and November 15, 2018; accepted on December 10, 2018

Particle filters are broadly used to approximate posterior distributions of hidden states in state-space models by means of sets of weighted particles. While the convergence of the filter is guaranteed when the number of particles tends to infinity, the quality of the approximation is usually unknown but strongly dependent on the number of particles. In this paper, we propose a novel method for assessing the convergence of particle filters online manner, as well as a simple scheme for the online adaptation of the number of particles based on the convergence assessment. The method is based on a sequential comparison between the actual observations and their predictive probability distributions approximated by the filter. We provide a rigorous theoretical analysis of the proposed methodology and, as an example of its practical use, we present simulations of a simple algorithm for the dynamic and online adaption of the number of particles during the operation of a particle filter on a stochastic version of the Lorenz system.

We consider mesh networks composed of groups of relaying nodes which operate in decode-and-forward mode, where each node from a group relays information to all the nodes in the next group. We study these networks in two setups, one where the nodes have complete channel state information from the nodes that transmit to them, and another when they only have the statistics of the channel. We derive recursive expressions for the probabilities of errors of the nodes and present several implementations of detectors used in these networks. We compare the mesh networks with multihop networks, the latter being formed by a set of parallel sections of multiple relaying nodes. We demonstrate with numerous simulations that there are significant improvements in performance of mesh over multihop networks in various scenarios., Comment: 23 pages, 5 figures

We present a consensus-based distributed particle filter (PF) for wireless sensor networks. Each sensor runs a local PF to compute a global state estimate that takes into account the measurements of all sensors. The local PFs use the joint (all-sensors) likelihood function, which is calculated in a distributed way by a novel generalization of the likelihood consensus scheme. A performance improvement (or a reduction of the required number of particles) is achieved by a novel distributed, consensus-based method for adapting the proposal densities of the local PFs. The performance of the proposed distributed PF is demonstrated for a target tracking problem.

We consider distributed state estimation in a wireless sensor network without a fusion center. Each sensor performs a global estimation task---based on the past and current measurements of all sensors---using only local processing and local communications with its neighbors. In this estimation task, the joint (all-sensors) likelihood function (JLF) plays a central role as it epitomizes the measurements of all sensors. We propose a distributed method for computing, at each sensor, an approximation of the JLF by means of consensus algorithms. This "likelihood consensus" method is applicable if the local likelihood functions of the various sensors (viewed as conditional probability density functions of the local measurements) belong to the exponential family of distributions. We then use the likelihood consensus method to implement a distributed particle filter and a distributed Gaussian particle filter. Each sensor runs a local particle filter, or a local Gaussian particle filter, that computes a global state estimate. The weight update in each local (Gaussian) particle filter employs the JLF, which is obtained through the likelihood consensus scheme. For the distributed Gaussian particle filter, the number of particles can be significantly reduced by means of an additional consensus scheme. Simulation results are presented to assess the performance of the proposed distributed particle filters for a multiple target tracking problem.