Search

The Information Bottleneck (IB) method is an information theoretical framework to design a parsimonious and tunable feature-extraction mechanism, such that the extracted features are maximally relevant to a specific learning or inference task. Despite its theoretical value, the IB is based on a functional optimization problem that admits a closed form solution only on specific cases (e.g., Gaussian distributions), making it difficult to be applied in most applications, where it is necessary to resort to complex and approximated variational implementations. To overcome this limitation, we propose an approach to adapt the closed-form solution of the Gaussian IB to a general task. Whichever is the inference task to be performed by a (possibly deep) neural-network, the key idea is to opportunistically design a regression sub-task, embedded in the original problem, where we can safely assume a (joint) multivariate normality between the sub-task's inputs and outputs. In this way we can exploit a fixed and pre-trained neural network to process the input data, using a tunable number of features, to trade data-size and complexity for accuracy. This approach is particularly useful every time a device needs to transmit data (or features) to a server that has to fulfil an inference task, as it provides a principled way to extract the most relevant features for the task to be executed, while looking for the best trade-off between the size of the feature vector to be transmitted, inference accuracy, and complexity. Extensive simulation results testify the effectiveness of the proposed method and encourage to further investigate this research line., Comment: To appear in IEEE Open Journal of the Communications Society, 2024

Deep Neural Network (DNN) splitting is one of the key enablers of edge Artificial Intelligence (AI), as it allows end users to pre-process data and offload part of the computational burden to nearby Edge Cloud Servers (ECSs). This opens new opportunities and degrees of freedom in balancing energy consumption, delay, accuracy, privacy, and other trustworthiness metrics. In this work, we explore the opportunity of DNN splitting at the edge of 6G wireless networks to enable low energy cooperative inference with target delay and accuracy with a goal-oriented perspective. Going beyond the current literature, we explore new trade-offs that take into account the accuracy degradation as a function of the Splitting Point (SP) selection and wireless channel conditions. Then, we propose an algorithm that dynamically controls SP selection, local computing resources, uplink transmit power and bandwidth allocation, in a goal-oriented fashion, to meet a target goal-effectiveness. To the best of our knowledge, this is the first work proposing adaptive SP selection on the basis of all learning performance (i.e., energy, delay, accuracy), with the aim of guaranteeing the accomplishment of a goal (e.g., minimize the energy consumption under latency and accuracy constraints). Numerical results show the advantages of the proposed SP selection and resource allocation, to enable energy frugal and effective edge AI., Comment: Submitted at EuCNC 2024

Internet of Things (IoT) applications combine sensing, wireless communication, intelligence, and actuation, enabling the interaction among heterogeneous devices that collect and process considerable amounts of data. However, the effectiveness of IoT applications needs to face the limitation of available resources, including spectrum, energy, computing, learning and inference capabilities. This paper challenges the prevailing approach to IoT communication, which prioritizes the usage of resources in order to guarantee perfect recovery, at the bit level, of the data transmitted by the sensors to the central unit. We propose a novel approach, called goal-oriented (GO) IoT system design, that transcends traditional bit-related metrics and focuses directly on the fulfillment of the goal motivating the exchange of data. The improvement is then achieved through a comprehensive system optimization, integrating sensing, communication, computation, learning, and control. We provide numerical results demonstrating the practical applications of our methodology in compelling use cases such as edge inference, cooperative sensing, and federated learning. These examples highlight the effectiveness and real-world implications of our proposed approach, with the potential to revolutionize IoT systems., Comment: Accepted for publication on IEEE Internet of Things Magazine, special issue on "Task-Oriented Communications and Networking for the Internet of Things"

Edge Learning (EL) pushes the computational resources toward the edge of 5G/6G network to assist mobile users requesting delay-sensitive and energy-aware intelligent services. A common challenge in running inference tasks from remote is to extract and transmit only the features that are most significant for the inference task. From this perspective, EL can be effectively coupled with goal-oriented communications, whose aim is to transmit only the information {\it relevant} to perform the inference task, under prescribed accuracy, delay, and energy constraints. In this work, we consider a multi-user/single server wireless network, where the users can opportunistically decide whether to perform the inference task by themselves or, alternatively, to offload the data to the edge server for remote processing. The data to be transmitted undergoes a goal-oriented compression stage performed using a convolutional encoder, jointly trained with a convolutional decoder running at the edge-server side. Employing Lyapunov optimization, we propose a method to jointly and dynamically optimize the selection of the most suitable encoding/decoding scheme, together with the allocation of computational and transmission resources, across all the users and the edge server. Extensive simulations confirm the effectiveness of the proposed approaches and highlight the trade-offs between energy, latency, and learning accuracy., Comment: 15 pages, 11 figures

Theoretical analysis of orthogonal frequency division multiplexing (OFDM) systems equipped at the receiver by a non-linear impulsive noise suppressor is a challenging topic in communication systems. Indeed, although an exact closed-form expression for the output signal-to-noise ratio (SNR) of such OFDM systems is available for widely used impulsive noise models, theoretical analysis of the associated symbol error rate (SER) is still open. So far, the analytical SER expressions available in the literature approximate the time-domain impulsive noise, as Gaussian distributed in the discrete frequency domain. Conversely, this work presents an accurate analysis of the distortion noise at the nonlinearity output exploiting a Gaussian mixture model (GMM). By using GMMs we unified the approach of SER prediction for unmitigated systems, as well as for the mitigated ones, equipped by non-linear impulsive noise suppressors, including blanking, clipping, clipping-blanking and attenuating processors. Closed form expressions for the SER are derived both for non-fading and frequency-selective Rayleigh and Rician fading channels affected by impulsive noise which is represented by GMMs, thus including Bernoulli-Gaussian (BG), Middleton Class-A, as well as (approximated) alpha-stable noise. Theoretic SER performance are compared with simulations, showing very good agreement for all the impulsive noise scenarios and the non-linear suppressors.

One of the most crucial challenges in graph signal processing is the sampling of bandlimited graph signals, i.e., signals that are sparse in a well-defined graph Fourier domain. So far, the prior art is mostly focused on (sub)sampling single snapshots of graph signals ignoring their evolution over time. However, time can bring forth new insights, since many real signals like sensor measurements, biological, financial, and network signals in general, have intrinsic correlations in both domains. In this work, {we fill this lacuna} by jointly considering the graph-time nature of graph signals, named \emph{graph processes} for two main tasks: \emph{i)} observability of graph processes; and \emph{ii)} tracking of graph processes via Kalman filtering; both from a (possibly time-varying) subset of nodes. A detailed mathematical analysis ratifies the proposed methods and provides insights into the role played by the different actors, such as the graph topology, the process bandwidth, and the sampling strategy. Moreover, (sub)optimal sampling strategies that jointly exploit the nature of the graph structure and graph process are proposed. Several numerical tests on both synthetic and real data validate our theoretical findings and illustrate the performance of the proposed methods in coping with time-varying graph signals., Comment: Submitted to Transactions on Signal Processing

The goal of this paper is to propose novel strategies for adaptive learning of signals defined over graphs, which are observed over a (randomly time-varying) subset of vertices. We recast two classical adaptive algorithms in the graph signal processing framework, namely, the least mean squares (LMS) and the recursive least squares (RLS) adaptive estimation strategies. For both methods, a detailed mean-square analysis illustrates the effect of random sampling on the adaptive reconstruction capability and the steady-state performance. Then, several probabilistic sampling strategies are proposed to design the sampling probability at each node in the graph, with the aim of optimizing the tradeoff between steady-state performance, graph sampling rate, and convergence rate of the adaptive algorithms. Finally, a distributed RLS strategy is derived and is shown to be convergent to its centralized counterpart. Numerical simulations carried out over both synthetic and real data illustrate the good performance of the proposed sampling and reconstruction strategies for (possibly distributed) adaptive learning of signals defined over graphs., Comment: Submitted to IEEE Transactions on Signal Processing, September 2017

Finite impulse response (FIR) graph filters play a crucial role in the field of signal processing on graphs. However, when the graph signal is time-varying, the state of the art FIR graph filters do not capture the time variations of the input signal. In this work, we propose an extension of FIR graph filters to capture also the signal variations over time. By considering also the past values of the graph signal, the proposed FIR graph filter extends naturally to a 2-dimensional filter, capturing jointly the signal variations over the graph and time. As a particular case of interest we focus on 2-dimensional separable graph-temporal filters, which can be implemented in a distributed fashion at the price of higher communication costs. This allows us to give filter specifications and perform the design independently in the graph and temporal domain. The work is concluded by analyzing the proposed approach for stochastic graph signals, where the first and second order moments of the output signal are characterized., Comment: 5 pages, 4 figures, letter

The minimum mean-squared error (MMSE) is one of the most popular criteria for Bayesian estimation. Conversely, the signal-to-noise ratio (SNR) is a typical performance criterion in communications, radar, and generally detection theory. In this paper we first formalize an SNR criterion to design an estimator, and then we prove that there exists an equivalence between MMSE and maximum-SNR estimators, for any statistics. We also extend this equivalence to specific classes of suboptimal estimators, which are expressed by a basis expansion model (BEM). Then, by exploiting an orthogonal BEM for the estimator, we derive the MMSE estimator constrained to a given quantization resolution of the noisy observations, and we prove that this suboptimal MMSE estimator tends to the optimal MMSE estimator that uses an infinite resolution of the observation. Besides, we derive closed-form expressions for the mean-squared error (MSE) and for the SNR of the proposed suboptimal estimators, and we show that these expressions constitute tight, asymptotically exact, bounds for the optimal MMSE and maximum SNR., Comment: Submitted to IEEE Transactions on Signal Processing on Dec. 2, 2015; revised on May 19, 2016

The aim of this paper is to propose a least mean squares (LMS) strategy for adaptive estimation of signals defined over graphs. Assuming the graph signal to be band-limited, over a known bandwidth, the method enables reconstruction, with guaranteed performance in terms of mean-square error, and tracking from a limited number of observations over a subset of vertices. A detailed mean square analysis provides the performance of the proposed method, and leads to several insights for designing useful sampling strategies for graph signals. Numerical results validate our theoretical findings, and illustrate the performance of the proposed method. Furthermore, to cope with the case where the bandwidth is not known beforehand, we propose a method that performs a sparse online estimation of the signal support in the (graph) frequency domain, which enables online adaptation of the graph sampling strategy. Finally, we apply the proposed method to build the power spatial density cartography of a given operational region in a cognitive network environment., Comment: Submitted to IEEE Transactions on Signal and Information Processing over Networks

5G cellular communications promise to deliver the gigabit experience to mobile users, with a capacity increase of up to three orders of magnitude with respect to current LTE systems. There is widespread agreement that such an ambitious goal will be realized through a combination of innovative techniques involving different network layers. At the physical layer, the OFDM modulation format, along with its multiple-access strategy OFDMA, is not taken for granted, and several alternatives promising larger values of spectral efficiency are being considered. This paper provides a review of some modulation formats suited for 5G, enriched by a comparative analysis of their performance in a cellular environment, and by a discussion on their interactions with specific 5G ingredients. The interaction with a massive MIMO system is also discussed by employing real channel measurements., Comment: to appear IEEE Signal Processing Magazine, special issue on Signal Processing for the 5G Revolution, November 2014

The paper focuses on minimum mean square error (MMSE) Bayesian estimation for a Gaussian source impaired by additive Middleton's Class-A impulsive noise. In addition to the optimal Bayesian estimator, the paper considers also the soft-limiter and the blanker, which are two popular suboptimal estimators characterized by very low complexity. The MMSE-optimum thresholds for such suboptimal estimators are obtained by practical iterative algorithms with fast convergence. The paper derives also the optimal thresholds according to a maximum-SNR (MSNR) criterion, and establishes connections with the MMSE criterion. Furthermore, closed form analytic expressions are derived for the MSE and the SNR of all the suboptimal estimators, which perfectly match simulation results. Noteworthy, these results can be applied to characterize the receiving performance of any multicarrier system impaired by a Gaussian-mixture noise, such as asymmetric digital subscriber lines (ADSL) and power-line communications (PLC)., Comment: 30 pages, 13 figures, part of this work has been submitted to IEEE Signal Processing Letters

This paper investigates the statistical properties of non-linear transformations (NLT) of random variables, in order to establish useful tools for estimation and information theory. Specifically, the paper focuses on linear regression analysis of the NLT output and derives sufficient general conditions to establish when the input-output regression coefficient is equal to the \emph{partial} regression coefficient of the output with respect to a (additive) part of the input. A special case is represented by zero-mean Gaussian inputs, obtained as the sum of other zero-mean Gaussian random variables. The paper shows how this property can be generalized to the regression coefficient of non-linear transformations of Gaussian-mixtures. Due to its generality, and the wide use of Gaussians and Gaussian-mixtures to statistically model several phenomena, this theoretical framework can find applications in multiple disciplines, such as communication, estimation, and information theory, when part of the nonlinear transformation input is the quantity of interest and the other part is the noise. In particular, the paper shows how the said properties can be exploited to simplify closed-form computation of the signal-to-noise ratio (SNR), the estimation mean-squared error (MSE), and bounds on the mutual information in additive non-Gaussian (possibly non-linear) channels, also establishing relationships among them., Comment: 26 pages, 4 figures (8 sub-figures), submitted to IEEE Trans. on Information Theory 20th April 2013