Search

A fundamental problem in decentralized networked systems is to coordinate actions of different agents so that they reach a state of agreement. In such applications, it is additionally desirable that the actions at various nodes may not be anticipated by malicious eavesdroppers. Motivated by this, we investigate the problem of secure multi-terminal strong coordination aided by a multiple-access wiretap channel. In this setup, independent and identically distributed copies of correlated sources are observed by two transmitters who encode the channel inputs to the MAC-WT. The legitimate receiver observing the channel output and side information correlated with the sources must produce approximately i.i.d. copies of an output variable jointly distributed with the sources. Furthermore, we demand that an external eavesdropper learns essentially nothin g about the sources and the simulated output sequence by observing its own MAC-WT output. This setting is aided by the presence of independent pairwise shared randomness between each encoder and the legitimate decoder, that is unavailable to the eavesdropper. We derive an achievable rate region based on a combination of coordination coding and wiretap coding, along with an outer bound. The inner bound is shown to be tight and a complete characterization is derived for the special case when the sources are conditionally independent given the decoder side information and the legitimate channel is composed of deterministic links. Further, we also analyze a more general scenario with possible encoder cooperation, where one of the encoders can non-causally crib from the other encoders input, for which an achievable rate region is proposed. We then explicitly compute the rate regions for an example both with and without cribbing between the encoders, and demonstrate that cribbing strictly improves upon the achievable rate region., Comment: Extended version of ISIT 2024 paper

In this paper, we investigate the performance of an integrated sensing and communication (ISAC) system within a cell-free massive multiple-input multiple-output (MIMO) system. Each access point (AP) operates in the millimeter-wave (mmWave) frequency band. The APs jointly serve the user equipments (UEs) in the downlink while simultaneously detecting a target through dedicated sensing beams, which are directed toward a reconfigurable intelligent surface (RIS). Although the AP-RIS, RIS-target, and AP-target channels have both line-of-sight (LoS) and non-line-of-sight (NLoS) parts, it is assumed only knowledge of the LoS paths is available. A key contribution of this study is the consideration of clutter, which degrades the target detection if not handled. We propose an algorithm to alternatively optimize the transmit power allocation and the RIS phase-shift matrix, maximizing the target signal-to-clutter-plus-noise ratio (SCNR) while ensuring a minimum signal-to-interference-plus-noise ratio (SINR) for the UEs. Numerical results demonstrate that exploiting clutter subspace significantly enhances detection probability, particularly at high clutter-to-noise ratios, and reveal that an increased number of transmit side clusters impair detection performance. Finally, we highlight the performance gains achieved using a dedicated sensing stream., Comment: submitted to IEEE ICC25

We study agents acting in an unknown environment where the agent's goal is to find a robust policy. We consider robust policies as policies that achieve high cumulative rewards for all possible environments. To this end, we consider agents minimizing the maximum regret over different environment parameters, leading to the study of minimax regret. This research focuses on deriving information-theoretic bounds for minimax regret in Markov Decision Processes (MDPs) with a finite time horizon. Building on concepts from supervised learning, such as minimum excess risk (MER) and minimax excess risk, we use recent bounds on the Bayesian regret to derive minimax regret bounds. Specifically, we establish minimax theorems and use bounds on the Bayesian regret to perform minimax regret analysis using these minimax theorems. Our contributions include defining a suitable minimax regret in the context of MDPs, finding information-theoretic bounds for it, and applying these bounds in various scenarios.

In this paper, we present SCOREQ, a novel approach for speech quality prediction. SCOREQ is a triplet loss function for contrastive regression that addresses the domain generalisation shortcoming exhibited by state of the art no-reference speech quality metrics. In the paper we: (i) illustrate the problem of L2 loss training failing at capturing the continuous nature of the mean opinion score (MOS) labels; (ii) demonstrate the lack of generalisation through a benchmarking evaluation across several speech domains; (iii) outline our approach and explore the impact of the architectural design decisions through incremental evaluation; (iv) evaluate the final model against state of the art models for a wide variety of data and domains. The results show that the lack of generalisation observed in state of the art speech quality metrics is addressed by SCOREQ. We conclude that using a triplet loss function for contrastive regression improves generalisation for speech quality prediction models but also has potential utility across a wide range of applications using regression-based predictive models., Comment: Accepted NeurIPS 2024

This paper provides an overview of a problem in information-theoretic privacy mechanism design, addressing two scenarios in which private data is either observable or hidden. In each scenario, different privacy measures are used, including bounded mutual information and two types of per-letter privacy constraints. Considering the first scenario, an agent observes useful data that is correlated with private data, and wants to disclose the useful information to a user. Due to the privacy concerns, direct disclosure is prohibited. Hence, a privacy mechanism is designed to generate disclosed data which maximizes the revealed information about the useful data while satisfying a privacy constraint. In the second scenario, the agent has additionally access to the private data. We discuss how the Functional Representation Lemma, the Strong Functional Representation Lemma, and their extended versions are useful for designing low-complexity privacy mechanisms that achieve optimal privacy-utility trade-offs under certain constraints. Furthermore, another privacy design problem is presented where part of the private attribute is more private than the remaining part. Finally, we provide applications including semantic communications, caching and delivery, and compression designs, where the approach can be applied., Comment: arXiv admin note: text overlap with arXiv:2212.12475

Integrated sensing and communications (ISAC) is a promising component of 6G networks, fusing communication and radar technologies to facilitate new services. Additionally, the use of extremely large-scale antenna arrays (ELLA) at the ISAC common receiver not only facilitates terahertz-rate communication links but also significantly enhances the accuracy of target detection in radar applications. In practical scenarios, communication scatterers and radar targets often reside in close proximity to the ISAC receiver. This, combined with the use of ELLA, fundamentally alters the electromagnetic characteristics of wireless and radar channels, shifting from far-field planar-wave propagation to near-field spherical wave propagation. Under the far-field planar-wave model, the phase of the array response vector varies linearly with the antenna index. In contrast, in the near-field spherical wave model, this phase relationship becomes nonlinear. This shift presents a fundamental challenge: the widely-used Fourier analysis can no longer be directly applied for target detection and communication channel estimation at the ISAC common receiver. In this work, we propose a feasible solution to address this fundamental issue. Specifically, we demonstrate that there exists a high-dimensional space in which the phase nonlinearity can be expressed as linear. Leveraging this insight, we develop a lifted super-resolution framework that simultaneously performs communication channel estimation and extracts target parameters with high precision.

A statistical cache-aided compression problem with a privacy constraint is studied, where a server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is linked through a shared channel to $K$ users, where each has access to a local cache memory of size $MF$ bits. During the placement phase, the server fills the users' caches without prior knowledge of their demands, while the delivery phase takes place after the users send their demands to the server. We assume that each file in database $Y_i$ is arbitrarily correlated with a private attribute $X$, and an adversary is assumed to have access to the shared channel. The users and the server have access to a shared key $W$. The goal is to design the cache contents and the delivered message $\cal C$ such that the average length of $\mathcal{C}$ is minimized, while satisfying: i. The response $\cal C$ does not reveal any information about $X$, i.e., $I(X;\mathcal{C})=0$; ii. User $i$ can decode its demand, $Y_{d_i}$, by using the shared key $W$, $\cal C$, and its local cache $Z_i$. In a previous work, we have proposed a variable-length coding scheme that combines privacy-aware compression with coded caching techniques. In this paper, we propose a new achievability scheme using minimum entropy coupling concept and a greedy entropy-based algorithm. We show that the proposed scheme improves the previous results. Moreover, considering two special cases we improve the obtained bounds using the common information concept.

In this article, we study the fundamental limits in the design of fair and/or private representations achieving perfect demographic parity and/or perfect privacy through the lens of information theory. More precisely, given some useful data $X$ that we wish to employ to solve a task $T$, we consider the design of a representation $Y$ that has no information of some sensitive attribute or secret $S$, that is, such that $I(Y;S) = 0$. We consider two scenarios. First, we consider a design desiderata where we want to maximize the information $I(Y;T)$ that the representation contains about the task, while constraining the level of compression (or encoding rate), that is, ensuring that $I(Y;X) \leq r$. Second, inspired by the Conditional Fairness Bottleneck problem, we consider a design desiderata where we want to maximize the information $I(Y;T|S)$ that the representation contains about the task which is not shared by the sensitive attribute or secret, while constraining the amount of irrelevant information, that is, ensuring that $I(Y;X|T,S) \leq r$. In both cases, we employ extended versions of the Functional Representation Lemma and the Strong Functional Representation Lemma and study the tightness of the obtained bounds. Every result here can also be interpreted as a coding with perfect privacy problem by considering the sensitive attribute as a secret.

We investigate the in-distribution generalization of machine learning algorithms. We depart from traditional complexity-based approaches by analyzing information-theoretic bounds that quantify the dependence between a learning algorithm and the training data. We consider two categories of generalization guarantees: 1) Guarantees in expectation: These bounds measure performance in the average case. Here, the dependence between the algorithm and the data is often captured by information measures. While these measures offer an intuitive interpretation, they overlook the geometry of the algorithm's hypothesis class. Here, we introduce bounds using the Wasserstein distance to incorporate geometry, and a structured, systematic method to derive bounds capturing the dependence between the algorithm and an individual datum, and between the algorithm and subsets of the training data. 2) PAC-Bayesian guarantees: These bounds measure the performance level with high probability. Here, the dependence between the algorithm and the data is often measured by the relative entropy. We establish connections between the Seeger--Langford and Catoni's bounds, revealing that the former is optimized by the Gibbs posterior. We introduce novel, tighter bounds for various types of loss functions. To achieve this, we introduce a new technique to optimize parameters in probabilistic statements. To study the limitations of these approaches, we present a counter-example where most of the information-theoretic bounds fail while traditional approaches do not. Finally, we explore the relationship between privacy and generalization. We show that algorithms with a bounded maximal leakage generalize. For discrete data, we derive new bounds for differentially private algorithms that guarantee generalization even with a constant privacy parameter, which is in contrast to previous bounds in the literature.

This paper characterizes the trade-offs between information and energy transmission over an additive white Gaussian noise channel in the finite block-length regime with finite channel input symbols. These trade-offs are characterized in the form of inequalities involving the information transmission rate, energy transmission rate, decoding error probability (DEP) and energy outage probability (EOP) for a given finite block-length code. The first set of results identify the set of necessary conditions that a given code must satisfy for simultaneous information and energy transmission. Following this, a novel method for constructing a family of codes that can satisfy a target information rate, energy rate, DEP and EOP is proposed. Finally, the achievability results identify the set of tuples of information rate, energy rate, DEP and EOP that can be simultaneously achieved by the constructed family of codes., Comment: arXiv admin note: substantial text overlap with arXiv:2211.05873

This paper explores the integration of model-based and data-driven approaches within the realm of neural speech and audio coding systems. It highlights the challenges posed by the subjective evaluation processes of speech and audio codecs and discusses the limitations of purely data-driven approaches, which often require inefficiently large architectures to match the performance of model-based methods. The study presents hybrid systems as a viable solution, offering significant improvements to the performance of conventional codecs through meticulously chosen design enhancements. Specifically, it introduces a neural network-based signal enhancer designed to post-process existing codecs' output, along with the autoencoder-based end-to-end models and LPCNet--hybrid systems that combine linear predictive coding (LPC) with neural networks. Furthermore, the paper delves into predictive models operating within custom feature spaces (TF-Codec) or predefined transform domains (MDCTNet) and examines the use of psychoacoustically calibrated loss functions to train end-to-end neural audio codecs. Through these investigations, the paper demonstrates the potential of hybrid systems to advance the field of speech and audio coding by bridging the gap between traditional model-based approaches and modern data-driven techniques., Comment: Accepted for publication in IEEE Signal Processing Magazine

Hyperspherical Prototypical Learning (HPL) is a supervised approach to representation learning that designs class prototypes on the unit hypersphere. The prototypes bias the representations to class separation in a scale invariant and known geometry. Previous approaches to HPL have either of the following shortcomings: (i) they follow an unprincipled optimisation procedure; or (ii) they are theoretically sound, but are constrained to only one possible latent dimension. In this paper, we address both shortcomings. To address (i), we present a principled optimisation procedure whose solution we show is optimal. To address (ii), we construct well-separated prototypes in a wide range of dimensions using linear block codes. Additionally, we give a full characterisation of the optimal prototype placement in terms of achievable and converse bounds, showing that our proposed methods are near-optimal., Comment: 14 pages: 9 of the main paper, 2 of references, and 3 of appendices. To appear in the Proceedings of the Geometry-grounded Representation Learning and Generative Modeling at the 41st International Conference on Machine Learning, Vienna, Austria. Code is available at: https://github.com/martinlindstrom/coding_theoretic_hpl

This paper explores the implications of guaranteeing privacy by imposing a lower bound on the information density between the private and the public data. We introduce an operationally meaningful privacy measure called pointwise maximal cost (PMC) and demonstrate that imposing an upper bound on PMC is equivalent to enforcing a lower bound on the information density. PMC quantifies the information leakage about a secret to adversaries who aim to minimize non-negative cost functions after observing the outcome of a privacy mechanism. When restricted to finite alphabets, PMC can equivalently be defined as the information leakage to adversaries aiming to minimize the probability of incorrectly guessing randomized functions of the secret. We study the properties of PMC and apply it to standard privacy mechanisms to demonstrate its practical relevance. Through a detailed examination, we connect PMC with other privacy measures that impose upper or lower bounds on the information density. Our results highlight that lower bounding the information density is a more stringent requirement than upper bounding it. Overall, our work significantly bridges the gaps in understanding the relationships between various privacy frameworks and provides insights for selecting a suitable framework for a given application.

To enhance straggler resilience in federated learning (FL) systems, a semi-decentralized approach has been recently proposed, enabling collaboration between clients. Unlike the existing semi-decentralized schemes, which adaptively adjust the collaboration weight according to the network topology, this letter proposes a deterministic coded network that leverages wireless diversity for semi-decentralized FL without requiring prior information about the entire network. Furthermore, the theoretical analyses of the outage and the convergence rate of the proposed scheme are provided. Finally, the superiority of our proposed method over benchmark methods is demonstrated through comprehensive simulations.

In the near future, the Internet of Things will interconnect billions of devices, forming a vast network where users sporadically transmit short messages through multi-path wireless channels. These channels are characterized by the superposition of a small number of scaled and delayed copies of Dirac spikes. At the receiver, the observed signal is a sum of these convolved signals, and the task is to find the amplitudes, continuous-indexed delays, and transmitted messages from a single signal. This task is inherently ill-posed without additional assumptions on the channel or messages. In this work, we assume the channel exhibits sparsity in the delay domain and that i.i.d. random linear encoding is applied to the messages at the devices. Leveraging these assumptions, we propose a semidefinite programming optimization capable of simultaneously recovering both messages and the delay parameters of the channels from only a single received signal. Our theoretical analysis establishes that the required number of samples at the receiver scales proportionally to the sum-product of sparsity and message length of all users, aligning with the degrees of freedom in the proposed convex optimization framework. Numerical experiments confirm the efficacy of the proposed method in accurately estimating closely-spaced delay parameters and recovering messages.

We study a multi-task private semantic communication problem, in which an encoder has access to an information source arbitrarily correlated with some latent private data. A user has $L$ tasks with priorities. The encoder designs a message to be revealed which is called the semantic of the information source. Due to the privacy constraints the semantic can not be disclosed directly and the encoder adds noise to produce disclosed data. The goal is to design the disclosed data that maximizes the weighted sum of the utilities achieved by the user while satisfying a privacy constraint on the private data. In this work, we first consider a single-task scenario and design the added noise utilizing various methods including the extended versions of the Functional Representation Lemma, Strong Functional Representation Lemma, and separation technique. We then study the multi-task scenario and derive a simple design of the source semantics. We show that in the multi-task scenario the main problem can be divided into multiple parallel single-task problems.

In this work, we address the challenge of accurately obtaining channel state information at the transmitter (CSIT) for frequency division duplexing (FDD) multiple input multiple output systems. Although CSIT is vital for maximizing spatial multiplexing gains, traditional CSIT estimation methods often suffer from impracticality due to the substantial training and feedback overhead they require. To address this challenge, we leverage two sources of prior information simultaneously: the presence of limited local scatterers at the base station (BS) and the time-varying characteristics of the channel. The former results in a redundant angular sparsity of users' channels exceeding the spatial dimension (i.e., the number of BS antennas), while the latter provides a prior non-uniform distribution in the angular domain. We propose a weighted optimization framework that simultaneously reflects both of these features. The optimal weights are then obtained by minimizing the expected recovery error of the optimization problem. This establishes an analytical closed-form relationship between the optimal weights and the angular domain characteristics. Numerical experiments verify the effectiveness of our proposed approach in reducing the recovery error and consequently resulting in decreased training and feedback overhead.

This work considers an uplink wireless communication system where multiple users with multiple antennas transmit data frames over dynamic channels. Previous studies have shown that multiple transmit and receive antennas can substantially enhance the sum-capacity of all users when the channel is known at the transmitter and in the case of uncorrelated transmit and receive antennas. However, spatial correlations stemming from close proximity of transmit antennas and channel variation between pilot and data time slots, known as channel aging, can substantially degrade the transmission rate if they are not properly into account. In this work, we provide an analytical framework to concurrently exploit both of these features. Specifically, we first propose a beamforming framework to capture spatial correlations. Then, based on random matrix theory tools, we introduce a deterministic expression that approximates the average sum-capacity of all users. Subsequently, we obtain the optimal values of pilot spacing and beamforming vectors upon maximizing this expression. Simulation results show the impacts of path loss, velocity of mobile users and Rician factor on the resulting sum-capacity and underscore the efficacy of our methodology compared to prior works.

In this work, we consider the matrix completion problem, where the objective is to reconstruct a low-rank matrix from a few observed entries. A commonly employed approach involves nuclear norm minimization. For this method to succeed, the number of observed entries needs to scale at least proportional to both the rank of the ground-truth matrix and the coherence parameter. While the only prior information is oftentimes the low-rank nature of the ground-truth matrix, in various real-world scenarios, additional knowledge about the ground-truth low-rank matrix is available. For instance, in collaborative filtering, Netflix problem, and dynamic channel estimation in wireless communications, we have partial or full knowledge about the signal subspace in advance. Specifically, we are aware of some subspaces that form multiple angles with the column and row spaces of the ground-truth matrix. Leveraging this valuable information has the potential to significantly reduce the required number of observations. To this end, we introduce a multi-weight nuclear norm optimization problem that concurrently promotes the low-rank property as well the information about the available subspaces. The proposed weights are tailored to penalize each angle corresponding to each basis of the prior subspace independently. We further propose an optimal weight selection strategy by minimizing the coherence parameter of the ground-truth matrix, which is equivalent to minimizing the required number of observations. Simulation results validate the advantages of incorporating multiple weights in the completion procedure. Specifically, our proposed multi-weight optimization problem demonstrates a substantial reduction in the required number of observations compared to the state-of-the-art methods., Comment: arXiv admin note: text overlap with arXiv:2111.00235

This work investigates a multi-user, multi-antenna uplink wireless system, where multiple users transmit signals to a base station. Previous research has explored the potential for linear growth in spectral efficiency by employing multiple transmit and receive antennas. This gain depends on the quality of channel state information and uncorrelated antennas. However, spatial correlations, arising from closely-spaced antennas, and channel aging effects, stemming from the difference between the channel at pilot and data time instances, can substantially counteract these benefits and degrade the transmission rate, especially in non-stationary environments. To address these challenges, this work introduces a real-time beamforming framework to compensate for the spatial correlation effect. A channel estimation scheme is then developed, leveraging temporal channel correlations and considering mobile device velocity and antenna spacing. Subsequently, an expression approximating the average spectral efficiency is obtained, dependent on pilot spacing, pilot and data powers, and beamforming vectors. By maximizing this expression, optimal parameters are identified. Numerical results reveal the effectiveness of the proposed approach compared to prior works. Moreover, optimal pilot spacing remains unaffected by interference components such as path loss and the velocity of interference users. The impact of interference components also diminishes with an increasing number of transmit antennas., Comment: arXiv admin note: text overlap with arXiv:2401.13368 by other authors

Distribution-preserving integrated sensing and communication with secure reconstruction is investigated in this paper. In addition to the distortion constraint, we impose another constraint on the distance between the reconstructed sequence distribution and the original state distribution to force the system to preserve the statistical property of the channel states. An inner bound of the distribution-preserving capacity-distortion region is provided with some capacity region results under special cases. A numerical example demonstrates the tradeoff between the communication rate, reconstruction distortion and distribution preservation. Furthermore, we consider the case that the reconstructed sequence should be kept secret from an eavesdropper who also observes the channel output. An inner bound of the tradeoff region and a capacity-achieving special case are presented., Comment: Accepted by ISIT2024

The increased importance of cybersecurity in autonomous machinery is becoming evident in the forestry domain. Forestry worksites are becoming more complex with the involvement of multiple systems and system of systems. Hence, there is a need to investigate how to address cybersecurity challenges for autonomous systems of systems in the forestry domain. Using a literature review and adapting standards from similar domains, as well as collaborative sessions with domain experts, we identify challenges towards CE-certified autonomous forestry machines focusing on cybersecurity and safety. Furthermore, we discuss the relationship between safety and cybersecurity risk assessment and their relation to AI, highlighting the need for a holistic methodology for their assurance.

Typical iterated filters, such as the iterated extended Kalman filter (IEKF), iterated unscented Kalman filter (IUKF), and iterated posterior linearization filter (IPLF), have been developed to improve the linearization point (or density) of the likelihood linearization in the well-known extended Kalman filter (EKF) and unscented Kalman filter (UKF). A shortcoming of typical iterated filters is that they do not treat the linearization of the transition model of the system. To remedy this shortcoming, we introduce dynamically iterated filters (DIFs), a unified framework for iterated linearization-based nonlinear filters that deals with nonlinearities in both the transition model and the likelihood, thereby constituting a generalization of the aforementioned iterated filters. We further establish a relationship between the general DIF and the approximate iterated Rauch-Tung-Striebel smoother. This relationship allows for a Gauss-Newton interpretation, which in turn enables explicit step-size correction, leading to damped versions of the DIFs. The developed algorithms, both damped and non-damped, are numerically demonstrated in three examples, showing superior mean-squared error as well as improved parameter tuning robustness as compared to the analogous standard iterated filters., Comment: 12 pages. Submitted to Journal of Advances in Information Fusion for possible publication

Integrated sensing and communication (ISAC) has already established itself as a promising solution to the spectrum scarcity problem, even more so when paired with a reconfigurable intelligent surface (RIS), as RISs can shape the propagation environment by adjusting their phase-shift coefficients. Albeit the potential performance gain, a RIS is also a potential security threat to the system. In this paper, we explore both the positive and negative sides of having a RIS in a multi-user multiple-input multiple-output (MIMO) ISAC network. We first develop an alternating optimization algorithm, obtaining the active and passive beamforming vectors that maximize the sensing signal-to-noise ratio (SNR) under minimum signal-to-interference-plus-noise ratio (SINR) constraints for the communication users and finite power budget. We also investigate the destructive potential of the RIS by devising a RIS phase-shift optimization algorithm that minimizes the sensing SNR while preserving the same minimum communication SINR previously guaranteed by the system. We further investigate the impact of the RIS's individual element failures on the system performance. The simulation results show that the RIS performance-boosting potential is as good as its destructive one and that both of our optimization strategies are hindered by the investigated impairments., Comment: submitted to IEEE ICC25

Stragglers' effects are known to degrade FL performance. In this paper, we investigate federated learning (FL) over wireless networks in the presence of communication stragglers, where the power-constrained clients collaboratively train a global model by iteratively optimizing a local objective function with their local datasets and transmitting local model updates to the central parameter server (PS) through fading channels. To tackle communication stragglers without dataset sharing or prior information about the network at PS, we propose cooperative gradient coding (CoGC) for semi-decentralized FL to enable the exact global model recovery at PS. Furthermore, we conduct a thorough theoretical analysis of the proposed approach. Namely, an outage analysis of the proposed approach is provided, followed by a convergence analysis based on the failure probability of the global model recovery at PS. Nevertheless, simulation results reveal the superiority of the proposed approach in the presence of stragglers under imbalanced data distribution.

This paper studies the truncation method from Alquier [1] to derive high-probability PAC-Bayes bounds for unbounded losses with heavy tails. Assuming that the $p$-th moment is bounded, the resulting bounds interpolate between a slow rate $1 / \sqrt{n}$ when $p=2$, and a fast rate $1 / n$ when $p \to \infty$ and the loss is essentially bounded. Moreover, the paper derives a high-probability PAC-Bayes bound for losses with a bounded variance. This bound has an exponentially better dependence on the confidence parameter and the dependency measure than previous bounds in the literature. Finally, the paper extends all results to guarantees in expectation and single-draw PAC-Bayes. In order to so, it obtains analogues of the PAC-Bayes fast rate bound for bounded losses from [2] in these settings., Comment: 9 pages: 5 of main text, 1 of references, and 3 of appendices

In this article, we address the problem of federated learning in the presence of stragglers. For this problem, a coded federated learning framework has been proposed, where the central server aggregates gradients received from the non-stragglers and gradient computed from a privacy-preservation global coded dataset to mitigate the negative impact of the stragglers. However, when aggregating these gradients, fixed weights are consistently applied across iterations, neglecting the generation process of the global coded dataset and the dynamic nature of the trained model over iterations. This oversight may result in diminished learning performance. To overcome this drawback, we propose a new method named adaptive coded federated learning (ACFL). In ACFL, before the training, each device uploads a coded local dataset with additive noise to the central server to generate a global coded dataset under privacy preservation requirements. During each iteration of the training, the central server aggregates the gradients received from the non-stragglers and the gradient computed from the global coded dataset, where an adaptive policy for varying the aggregation weights is designed. Under this policy, we optimize the performance in terms of privacy and learning, where the learning performance is analyzed through convergence analysis and the privacy performance is characterized via mutual information differential privacy. Finally, we perform simulations to demonstrate the superiority of ACFL compared with the non-adaptive methods.

This paper considers the problem of distributed learning (DL) in the presence of stragglers. For this problem, DL methods based on gradient coding have been widely investigated, which redundantly distribute the training data to the workers to guarantee convergence when some workers are stragglers. However, these methods require the workers to transmit real-valued vectors during the process of learning, which induces very high communication burden. To overcome this drawback, we propose a novel DL method based on 1-bit gradient coding (1-bit GCDL), where 1-bit data encoded from the locally computed gradients are transmitted by the workers to reduce the communication overhead. We theoretically provide the convergence guarantees of the proposed method for both the convex loss functions and nonconvex loss functions. It is shown empirically that 1-bit GC-DL outperforms the baseline methods, which attains better learning performance under the same communication overhead., Comment: Accepted by IEEE TCOM

This paper studies the Bayesian regret of a variant of the Thompson-Sampling algorithm for bandit problems. It builds upon the information-theoretic framework of [Russo and Van Roy, 2015] and, more specifically, on the rate-distortion analysis from [Dong and Van Roy, 2020], where they proved a bound with regret rate of $O(d\sqrt{T \log(T)})$ for the $d$-dimensional linear bandit setting. We focus on bandit problems with a metric action space and, using a chaining argument, we establish new bounds that depend on the metric entropy of the action space for a variant of Thompson-Sampling. Under suitable continuity assumption of the rewards, our bound offers a tight rate of $O(d\sqrt{T})$ for $d$-dimensional linear bandit problems., Comment: 15 pages: 8 of main text and 7 of appendices

In this paper we establish existence, uniqueness, and boundedness results for an elliptic variational inequality coupled with a nonlinear ordinary differential equation. Under the general framework, we present a new application modelling the antiplane shear deformation of a static frictional adhesive contact problem. The adhesion process has been extensively studied, but it is usual to assume a priori that the intensity of adhesion is bounded by introducing truncation operators. The aim of this article is to remove this restriction. The proof is based on an iterative approximation scheme showing that the problem has a unique solution. A key ingredient is finding uniform a priori bounds for each iterate. These are obtained by adapting versions of the Moser iteration to our system of equations., Comment: 39 pages, 1 figure

Reconfigurable intelligent surfaces (RISs) have demonstrated significant potential for enhancing communication system performance if properly configured. However, a RIS might also pose a risk to the network security. In this letter, we explore the impact of a malicious RIS on a multi-user multiple-input single-output (MISO) system when the system is unaware of the RIS's malicious intentions. The objective of the malicious RIS is to degrade the \ac{SNR} of a specific \ac{UE}, with the option of preserving the SNR of the other UEs, making the attack harder to detect. To achieve this goal, we derive the optimal RIS phase-shift pattern, assuming perfect channel state information (CSI) at the hacker. We then relax this assumption by introducing CSI uncertainties and subsequently determine the RIS's phase-shift pattern using a robust optimization approach. Our simulations reveal a direct proportionality between the performance degradation caused by the malicious RIS and the number of reflective elements, along with resilience toward CSI uncertainties., Comment: Submitted to IEEE wireless communication letters

We examine the relationship between privacy metrics that utilize information density to measure information leakage between a private and a disclosed random variable. Firstly, we prove that bounding the information density from above or below in turn implies a lower or upper bound on the information density, respectively. Using this result, we establish new relationships between local information privacy, asymmetric local information privacy, pointwise maximal leakage and local differential privacy. We further provide applications of these relations to privacy mechanism design. Furthermore, we provide statements showing the equivalence between a lower bound on information density and risk-averse adversaries. More specifically, we prove an equivalence between a guessing framework and a cost-function framework that result in the desired lower bound on the information density.

In this paper, we consider a decentralized learning problem in the presence of stragglers. Although gradient coding techniques have been developed for distributed learning to evade stragglers, where the devices send encoded gradients with redundant training data, it is difficult to apply those techniques directly to decentralized learning scenarios. To deal with this problem, we propose a new gossip-based decentralized learning method with gradient coding (GOCO). In the proposed method, to avoid the negative impact of stragglers, the parameter vectors are updated locally using encoded gradients based on the framework of stochastic gradient coding and then averaged in a gossip-based manner. We analyze the convergence performance of GOCO for strongly convex loss functions. And we also provide simulation results to demonstrate the superiority of the proposed method in terms of learning performance compared with the baseline methods.

Several previous works have addressed the inherent trade-off between allocating resources in the power and time domains to pilot and data signals in multiple input multiple output systems over block-fading channels. In particular, when the channel changes rapidly in time, channel aging degrades the performance in terms of spectral efficiency without proper pilot spacing and power control. Despite recognizing non-stationary stochastic processes as more accurate models for time-varying wireless channels, the problem of pilot spacing and power control in multi-antenna systems operating over non-stationary channels is not addressed in the literature. In this paper, we address this gap by introducing a refined first-order autoregressive model that exploits the inherent temporal correlations over non-stationary Rician aging channels. We design a multi-frame structure for data transmission that better reflects the non-stationary fading environment than previously developed single-frame structures. Subsequently, to determine optimal pilot spacing and power control within this multi-frame structure, we develop an optimization framework and an efficient algorithm based on maximizing a deterministic equivalent expression for the spectral efficiency, demonstrating its generality by encompassing previous channel aging results. Our numerical results indicate the efficacy of the proposed method in terms of spectral efficiency gains over the single frame structure.

In this paper, we study a cell-free multiple-input multiple-output network equipped with integrated sensing and communication (ISAC) access points (APs). The distributed APs are used to jointly serve the communication needs of user equipments (UEs) while sensing a target, assumed to be an eavesdropper (Eve). To increase the system's robustness towards said Eve, we develop an ISAC waveform model that includes artificial noise (AN) aimed at degrading the Eve channel quality. The central processing unit receives the observations from each AP and calculates the optimal precoding and AN covariance matrices by solving a semi-definite relaxation of a constrained Cramer-Rao bound (CRB) minimization problem. Simulation results highlight an underlying trade-off between sensing and communication performances: in particular, the UEs signal-to-noise and interference ratio and the maximum Eve's signal to noise ratio are directly proportional to the CRB. Furthermore, the optimal AN covariance matrix is rank-1 and has a peak in the eve's direction, leading to a surprising inverse-proportionality between the UEs-Eve distance and optimal-CRB magnitude., Comment: Corrected version of the paper presented at WCNC24

Third-party dependency updates can cause a build to fail if the new dependency version introduces a change that is incompatible with the usage: this is called a breaking dependency update. Research on breaking dependency updates is active, with works on characterization, understanding, automatic repair of breaking updates, and other software engineering aspects. All such research projects require a benchmark of breaking updates that has the following properties: 1) it contains real-world breaking updates; 2) the breaking updates can be executed; 3) the benchmark provides stable scientific artifacts of breaking updates over time, a property we call reproducibility. To the best of our knowledge, such a benchmark is missing. To address this problem, we present BUMP, a new benchmark that contains reproducible breaking dependency updates in the context of Java projects built with the Maven build system. BUMP contains 571 breaking dependency updates collected from 153 Java projects. BUMP ensures long-term reproducibility of dependency updates on different platforms, guaranteeing consistent build failures. We categorize the different causes of build breakage in BUMP, providing novel insights for future work on breaking update engineering. To our knowledge, BUMP is the first of its kind, providing hundreds of real-world breaking updates that have all been made reproducible.

Over-the-air computation (AirComp) has emerged as a promising technology for fast wireless data aggregation by harnessing the superposition property of wireless multiple-access channels. This paper investigates a fluid antenna (FA) array-enhanced AirComp system, employing the new degrees of freedom achieved by antenna movements. Specifically, we jointly optimize the transceiver design and antenna position vector (APV) to minimize the mean squared error (MSE) between target and estimated function values. To tackle the resulting highly non-convex problem, we adopt an alternating optimization technique to decompose it into three subproblems. These subproblems are then iteratively solved until convergence, leading to a locally optimal solution. Numerical results show that FA arrays with the proposed transceiver and APV design significantly outperform the traditional fixed-position antenna arrays in terms of MSE.

In this paper, we investigate the communication designs of over-the-air computation (AirComp) empowered federated learning (FL) systems considering uplink model aggregation and downlink model dissemination jointly. We first derive an upper bound on the expected difference between the training loss and the optimal loss, which reveals that optimizing the FL performance is equivalent to minimizing the distortion in the received global gradient vector at each edge node. As such, we jointly optimize each edge node transmit and receive equalization coefficients along with the edge server forwarding matrix to minimize the maximum gradient distortion across all edge nodes. We further utilize the MNIST dataset to evaluate the performance of the considered FL system in the context of the handwritten digit recognition task. Experiment results show that deploying multiple antennas at the edge server significantly reduces the distortion in the received global gradient vector, leading to a notable improvement in recognition accuracy compared to the single antenna case., Comment: This paper has been accepted by the IEEE 98th Vehicular Technology Conference (VTC2023-Fall), Hong Kong, China, 2023

In this paper, we propose leveraging the active reconfigurable intelligence surface (RIS) to support reliable gradient aggregation for over-the-air computation (AirComp) enabled federated learning (FL) systems. An analysis of the FL convergence property reveals that minimizing gradient aggregation errors in each training round is crucial for narrowing the convergence gap. As such, we formulate an optimization problem, aiming to minimize these errors by jointly optimizing the transceiver design and RIS configuration. To handle the formulated highly non-convex problem, we devise a two-layer alternative optimization framework to decompose it into several convex subproblems, each solvable optimally. Simulation results demonstrate the superiority of the active RIS in reducing gradient aggregation errors compared to its passive counterpart., Comment: This paper was submitted to the IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN), Stockholm, Sweden, 2024

A private compression design problem is studied, where an encoder observes useful data $Y$, wishes to compress it using variable length code and communicates it through an unsecured channel. Since $Y$ is correlated with private data $X$, the encoder uses a private compression mechanism to design encoded message $\cal C$ and sends it over the channel. An adversary is assumed to have access to the output of the encoder, i.e., $\cal C$, and tries to estimate $X$. Furthermore, it is assumed that both encoder and decoder have access to a shared secret key $W$. In this work, we generalize the perfect privacy (secrecy) assumption and consider a non-zero leakage between the private data $X$ and encoded message $\cal C$. The design goal is to encode message $\cal C$ with minimum possible average length that satisfies non-perfect privacy constraints. We find upper and lower bounds on the average length of the encoded message using different privacy metrics and study them in special cases. For the achievability we use two-part construction coding and extended versions of Functional Representation Lemma. Lastly, in an example we show that the bounds can be asymptotically tight., Comment: arXiv admin note: text overlap with arXiv:2306.13184

A private compression design problem is studied, where an encoder observes useful data $Y$, wishes to compress it using variable length code and communicates it through an unsecured channel. Since $Y$ is correlated with private attribute $X$, the encoder uses a private compression mechanism to design encoded message $\cal C$ and sends it over the channel. An adversary is assumed to have access to the output of the encoder, i.e., $\cal C$, and tries to estimate $X$. Furthermore, it is assumed that both encoder and decoder have access to a shared secret key $W$. The design goal is to encode message $\cal C$ with minimum possible average length that satisfies a perfect privacy constraint. To do so we first consider two different privacy mechanism design problems and find upper bounds on the entropy of the optimizers by solving a linear program. We use the obtained optimizers to design $\cal C$. In two cases we strengthen the existing bounds: 1. $|\mathcal{X}|\geq |\mathcal{Y}|$; 2. The realization of $(X,Y)$ follows a specific joint distribution. In particular, considering the second case we use two-part construction coding to achieve the upper bounds. Furthermore, in a numerical example we study the obtained bounds and show that they can improve the existing results., Comment: arXiv admin note: text overlap with arXiv:2306.13184, arXiv:2309.09034, arXiv:2211.15525, arXiv:2310.19122

The integrated sensing and communication (ISAC) problem with general state and channel distributions is investigated. General formulas of the capacity-distortion tradeoff for the ISAC problem under maximal and average distortion constraints are provided. The results cover some existing communication models such as the general point-to-point channel and Gel'fand-Pinsker channel. More details including memoryless states and channels, mixed states and channels, and rate-limited CSI at one side are considered. Numerical results focus on writing-on-dirty paper model and channel with ergodic/non-ergodic fading., Comment: 27 pages, 1 figure

This paper presents NOMAD (Non-Matching Audio Distance), a differentiable perceptual similarity metric that measures the distance of a degraded signal against non-matching references. The proposed method is based on learning deep feature embeddings via a triplet loss guided by the Neurogram Similarity Index Measure (NSIM) to capture degradation intensity. During inference, the similarity score between any two audio samples is computed through Euclidean distance of their embeddings. NOMAD is fully unsupervised and can be used in general perceptual audio tasks for audio analysis e.g. quality assessment and generative tasks such as speech enhancement and speech synthesis. The proposed method is evaluated with 3 tasks. Ranking degradation intensity, predicting speech quality, and as a loss function for speech enhancement. Results indicate NOMAD outperforms other non-matching reference approaches in both ranking degradation intensity and quality assessment, exhibiting competitive performance with full-reference audio metrics. NOMAD demonstrates a promising technique that mimics human capabilities in assessing audio quality with non-matching references to learn perceptual embeddings without the need for human-generated labels., Comment: Accepted for ICASSP 2024

A multi-user private data compression problem is studied. A server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected to an encoder. The encoder is connected through an unsecured link to a user. We assume that each file $Y_i$ is arbitrarily correlated with a private attribute $X$, which is assumed to be accessible by the encoder. Moreover, an adversary is assumed to have access to the link. The users and the encoder have access to a shared secret key $W$. We assume that at each time the user asks for a file $Y_{d_i}$, where $(d_1,\ldots,d_K)$ corresponds to the demand vector. The goal is to design the delivered message $\mathcal {C}=(\mathcal {C}_1,\ldots,\mathcal {C}_K)$ after the user send his demands to the encoder such that the average length of $\mathcal{C}$ is minimized, while satisfying: i. The message $\cal C$ does not reveal any information about $X$, i.e., $X$ and $\mathcal{C}$ are independent, which corresponds to the perfect privacy constraint; ii. The user is able to decode its demands, $Y_{d_i}$, by using $\cal C$, and the shared key $W$. Here, the encoder sequentially encode each demand $Y_{d_i}$ at time $i$, using the shared key and previous encoded messages. We propose a variable-length coding scheme that uses privacy-aware compression techniques. We study proposed upper and lower bounds on the average length of $\mathcal{C}$ in an example. Finally, we study an application considering cache-aided networks., Comment: arXiv admin note: substantial text overlap with arXiv:2306.13184

The design of a statistical signal processing privacy problem is studied where the private data is assumed to be observable. In this work, an agent observes useful data $Y$, which is correlated with private data $X$, and wants to disclose the useful information to a user. A statistical privacy mechanism is employed to generate data $U$ based on $(X,Y)$ that maximizes the revealed information about $Y$ while satisfying a privacy criterion. To this end, we use extended versions of the Functional Representation Lemma and Strong Functional Representation Lemma and combine them with a simple observation which we call separation technique. New lower bounds on privacy-utility trade-off are derived and we show that they can improve the previous bounds. We study the obtained bounds in different scenarios and compare them with previous results., Comment: arXiv admin note: substantial text overlap with arXiv:2201.08738, arXiv:2212.12475

This letter investigates relationships between iterated filtering algorithms based on statistical linearization, such as the iterated unscented Kalman filter (IUKF), and filtering algorithms based on quasi-Newton (QN) methods, such as the QN iterated extended Kalman filter (QN-IEKF). Firstly, it is shown that the IUKF and the iterated posterior linearization filter (IPLF) can be viewed as QN algorithms, by finding a Hessian correction in the QN-IEKF such that the IPLF iterate updates are identical to that of the QN-IEKF. Secondly, it is shown that the IPLF/IUKF update can be rewritten such that it is approximately identical to the QN-IEKF, albeit for an additional correction term. This enables a richer understanding of the properties of iterated filtering algorithms based on statistical linearization., Comment: 4 pages, Accepted to IEEE Signal Processing Letters