Search

Federated learning is a semi-distributed algorithm, where a server communicates with multiple dispersed clients to learn a global model. The federated architecture is not robust and is sensitive to communication and computational overloads due to its one-master multi-client structure. It can also be subject to privacy attacks targeting personal information on the communication links. In this work, we introduce graph federated learning (GFL), which consists of multiple federated units connected by a graph. We then show how graph homomorphic perturbations can be used to ensure the algorithm is di erentially private on the server level. While on the client level, we show that improvement in the di erentially private federated learning algorithm can be attained through the addition of random noise to the updates, as opposed to the models. We conduct both convergence and privacy theoretical analyses and illustrate performance by means of computer simulations.

Adaptive social learning is a useful tool for studying distributed decision-making problems over graphs. This paper investigates the effect of combination policies on the performance of adaptive social learning strategies. Using large-deviation analysis, it first derives a bound on the steady-state error probability and characterizes the optimal selection for the Perron eigenvectors of the combination policies. It subsequently studies the effect of the combination policy on the transient behavior of the learning strategy by estimating the adaptation time in the low signal-to-noise ratio regime. In the process, it is discovered that, interestingly, the influence of the combination policy on the transient behavior is insignificant, and thus it is more critical to employ policies that enhance the steady-state performance. The theoretical conclusions are illustrated by means of computer simulations.

This work proposes a multi-agent filtering algorithm over graphs for finite-state hidden Markov models (HMMs), which can be used for sequential state estimation or for tracking opinion formation over dynamic social networks. We show that the difference from the optimal centralized Bayesian solution is asymptotically bounded for geometrically ergodic transition models. Experiments illustrate the theoretical findings and in particular, demonstrate the superior performance of the proposed algorithm compared to a state-of-the-art social learning algorithm., Accepted to IEEE Data Science and Learning Workshop 2022

Driven by the need to solve increasingly complex optimization problems in signal processing and machine learning, there has been increasing interest in understanding the behavior of gradient-descent algorithms in non-convex environments. Most available works on distributed non-convex optimization problems focus on the deterministic setting where exact gradients are available at each agent. In this work and its Part II, we consider stochastic cost functions, where exact gradients are replaced by stochastic approximations and the resulting gradient noise persistently seeps into the dynamics of the algorithm. We establish that the diffusion learning strategy continues to yield meaningful estimates non-convex scenarios in the sense that the iterates by the individual agents will cluster in a small region around the network centroid. We use this insight to motivate a short-term model for network evolution over a finite-horizon. In Part II of this work, we leverage this model to establish descent of the diffusion strategy through saddle points in $O(1/\mu)$ steps, where $\mu$ denotes the step-size, and the return of approximately second-order stationary points in a polynomial number of iterations.

This work studies social learning under non-stationary conditions. Although designed for online inference, traditional social learning algorithms perform poorly under drifting conditions. To mitigate this drawback, we propose the Adaptive Social Learning (ASL) strategy. This strategy lever-ages an adaptive Bayesian update, where the adaptation degree can be modulated by tuning a suitable step-size parameter. The learning performance of the ASL algorithm is examined by means of a steady-state analysis. It is shown that, under the regime of small step-sizes: i) consistent learning is possible; ii) and an accurate prediction of the performance can be furnished in terms of a Gaussian approximation.

Federated learning involves a mixture of centralized and decentralized processing tasks, where a server regularly selects a sample of the agents and these in turn sample their local data to compute stochastic gradients for their learning updates. The sampling of both agents and data is generally uniform; however, in this work we consider non-uniform sampling. We derive optimal importance sampling strategies for both agent and data selection and show that under convexity and Lipschitz assumptions, non-uniform sampling without replacement improves the performance of the original FedAvg algorithm. We run experiments on a regression and classification problem to illustrate the theoretical results.

This work addresses the problem of sharing partial information within social learning strategies. In traditional social learning, agents solve a distributed multiple hypothesis testing problem by performing two operations at each instant: first, agents incorporate information from private observations to form their beliefs over a set of hypotheses; second, agents combine the entirety of their beliefs locally among neighbors. Within a sufficiently informative environment and as long as the connectivity of the network allows information to diffuse across agents, these algorithms enable agents to learn the true hypothesis. Instead of sharing the entirety of their beliefs, this work considers the case in which agents will only share their beliefs regarding one hypothesis of interest, with the purpose of evaluating its validity, and draws conditions under which this policy does not affect truth learning. We propose two approaches for sharing partial information, depending on whether agents behave in a self-aware manner or not. The results show how different learning regimes arise, depending on the approach employed and on the inherent characteristics of the inference problem. Furthermore, the analysis interestingly points to the possibility of deceiving the network, as long as the evaluated hypothesis of interest is close enough to the truth.

Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in general, in the sense that even simply verifying that a given point is a local minimum can be NP-hard [1]. Still, some relatively simple algorithms have been shown to lead to surprisingly good empirical results in many contexts of interest. Perhaps the most prominent example is the success of the backpropagation algorithm for training neural networks. Several recent works have pursued rigorous analytical justification for this phenomenon by studying the structure of the nonconvex optimization problems and establishing that simple algorithms, such as gradient descent and its variations, perform well in converging towards local minima and avoiding saddle-points. A key insight in these analyses is that gradient perturbations play a critical role in allowing local descent algorithms to efficiently distinguish desirable from undesirable stationary points and escape from the latter. In this article, we cover recent results on second-order guarantees for stochastic first-order optimization algorithms in centralized, federated, and decentralized architectures.

Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. While many federated learning architectures process data in an online manner, and are hence adaptive by nature, most performance analyses assume static optimization problems and offer no guarantees in the presence of drifts in the problem solution or data characteristics. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm. The results clarify the trade-off between convergence and tracking performance.

In social learning, agents form their opinions or beliefs about certain hypotheses by exchanging local information. This work considers the recent paradigm of weak graphs, where the network is partitioned into sending and receiving components, with the former having the possibility of exerting a domineering effect on the latter. Such graph structures are prevalent over social platforms. We will not be focusing on the direct social learning problem (which examines what agents learn), but rather on the dual or reverse learning problem (which examines how agents learned). Specifically, from observations of the stream of beliefs at certain agents, we would like to examine whether it is possible to learn the strength of the connections (influences) from sending components in the network to these receiving agents., To be presented at ICASSP 2020

Diffusion adaptation is a powerful strategy for distributed estimation and learning over networks. Motivated by the concept of combining adaptive filters, this work proposes a combination framework that aggregates the operation of multiple diffusion strategies for enhanced performance. By assigning a combination coefficient to each node, and using an adaptation mechanism to minimize the network error, we obtain a combined diffusion strategy that benefits from the best characteristics of all component strategies simultaneously in terms of excess-mean-square error (EMSE). Analyses of the universality are provided to show the superior performance of affine combination scheme and to characterize its behavior in the mean and mean-square sense. Simulation results are presented to demonstrate the effectiveness of the proposed strategies, as well as the accuracy of theoretical findings., 31 pages

Combining diffusion strategies with complementary properties enables enhanced performance when they can be run simultaneously. In this article, we first propose two schemes for the convex combination of two diffusion strategies, namely, the power-normalized scheme and the sign-regressor scheme. Then, we conduct theoretical analysis for one of the schemes, i.e., the power-normalized one. An analysis of universality shows that it cannot perform worse than any of its component strategies in terms of the excess mean-square-error (EMSE) at steady state, and sometimes even better. An analysis of stability also reveals that it is more stable than affine combination schemes already proposed by the authors in the literature. Next, several adjustments are proposed to further improve the performance of convex combination schemes. A discussion about the computational and communication complexity is provided, as well as a comparison between convex and affine combination schemes. Finally, simulation results are shown to demonstrate their effectiveness, the accuracy of the theoretical results, and the improved stability of the convex power-normalized scheme over the affine one.

International audience; The problem of learning simultaneously several related tasks has received considerable attention in several domains, especially in machine learning with the so-called multitask learning problem or learning to learn problem [1], [2]. Multitask learning is an approach to inductive transfer learning (using what is learned for one problem to assist in another problem) and helps improve generalization performance relative to learning each task separately by using the domain information contained in the training signals of related tasks as an inductive bias. Several strategies have been derived within this community under the assumption that all data are available beforehand at a fusion center. However, recent years have witnessed an increasing ability to collect data in a distributed and streaming manner. This requires the design of new strategies for learning jointly multiple tasks from streaming data over distributed (or networked) systems. This article provides an overview of multitask strategies for learning and adaptation over networks. The working hypothesis for these strategies is that agents are allowed to cooperate with each other in order to learn distinct, though related tasks. The article shows how cooperation steers the network limiting point and how different cooperation rules allow to promote different task relatedness models. It also explains how and when cooperation over multitask networks outperforms non-cooperative strategies.

We study the problem of distributed estimation over adaptive networks where communication delays exist between nodes. In particular, we investigate the diffusion Least-Mean- Square (LMS) strategy where delayed intermediate estimates (due to the communication channels) are employed during the combination step. One important question is: Do the delays affect the stability condition and performance? To answer this question, we conduct a detailed performance analysis in the mean and in the mean-square-error sense of the diffusion LMS with delayed estimates. Stability conditions, transient and steady-state mean-square-deviation (MSD) expressions are provided. One of the main findings is that diffusion LMS with delays can still converge under the same step-sizes condition of the diffusion LMS without delays. Finally, simulation results illustrate the theoretical findings., to appear in IEEE Signal Processing Letters

Adaptation and learning over multi-agent networks is a topic of great relevance with important implications. Elaborating on previous works on single-task networks engaged in decision problems, here we consider the multi-task version in the challenging scenario where the state of nature may change arbitrarily. We propose a data diffusion scheme for tracking these changes in real time, and investigate by numerical simulations the corresponding steady-state decision performance. For the slow-adaptation regime, the complete analytical characterization of the agents’ status is provided, under the simplifying assumption that the network connection matrix is correctly estimated.

This work studies the learning abilities of agents sharing partial beliefs over social networks. The agents observe data that could have risen from one of several hypotheses and interact locally to decide whether the observations they are receiving have risen from a particular hypothesis of interest. To do so, we establish the conditions under which it is sufficient to share partial information about the agents’ belief in relation to the hypothesis of interest. Some interesting convergence regimes arise.

This work studies a class of non-smooth decentralized multi-agent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common non-smooth term. We propose a general primal-dual algorithmic framework that unifies many existing state-of-the-art algorithms. We establish linear convergence of the proposed method to the exact solution in the presence of the non-smooth term. Moreover, for the more general class of problems with agent specific non-smooth terms, we show that linear convergence cannot be achieved (in the worst case) for the class of algorithms that uses the gradients and the proximal mappings of the smooth and non-smooth parts, respectively. We further provide a numerical counterexample that shows how some state-of-the-art algorithms fail to converge linearly for strongly-convex objectives and different local non-smooth terms., To appear in IEEE Transactions on Automatic Control

Part I of this paper considered optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that require the objectives across the network to lie in low-dimensional subspaces. Starting from the centralized projected gradient descent, an iterative and distributed solution was proposed that responds to streaming data and employs stochastic approximations in place of actual gradient vectors, which are generally unavailable. We examined the second-order stability of the learning algorithm and we showed that, for small step-sizes $\mu$, the proposed strategy leads to small estimation errors on the order of $\mu$. This Part II examines steady-state performance. The results reveal explicitly the influence of the gradient noise, data characteristics, and subspace constraints, on the network performance. The results also show that in the small step-size regime, the iterates generated by the distributed algorithm achieve the centralized steady-state performance., Comment: arXiv admin note: text overlap with arXiv:1905.08750

In this work, we revisit a classical incremental implementation of the primal-descent dual-ascent gradient method used for the solution of equality constrained optimization problems. We provide a short proof that establishes the linear (exponential) convergence of the algorithm for smooth strongly-convex cost functions and study its relation to the non-incremental implementation. We also study the effect of the augmented Lagrangian penalty term on the performance of distributed optimization algorithms for the minimization of aggregate cost functions over multi-agent networks. (C) 2020 Elsevier Ltd. All rights reserved.

This work studies the operation of multi-agent networks engaged in binary decision tasks, and derives performance expressions and performance operating curves under challenging conditions with some revealing insights. One of the main challenges in the analysis is that agents are only allowed to exchange one-bit messages, and the information at each agent therefore consists of both continuous and discrete components. Due to this mixed nature, the steady-state distribution of the state of each agent cannot be inferred from direct application of central limit arguments. Instead, the behavior of the continuous component is characterized in integral form by using a log-characteristic function, while the behavior of the discrete component is characterized by means of an asymmetric Bernoulli convolution. By exploiting these results, the article derives reliable approximate performance expressions for the network nodes that match well with the simulated results for a wide range of system parameters. The results also reveal an important interplay between continuous adaptation under constant step-size learning and the binary nature of the messages exchanged with neighbors., To appear in IEEE Trans. on Information Theory

In this chapter, we review the foundations of statistical inference over adaptive networks by considering two canonical problems: distributed estimation and distributed detection. In the former setting, agents cooperate to estimate a model of interest while in the second setting, the agents cooperate to detect a state of nature. We focus on adaptive learning solutions where agents are able to track drifts in the underlying models, and examine performance limits under both estimation and detection formulations. Special attention is paid to the detailed characterization of the steady-state performance. Certain universal laws are highlighted and compared against known laws for estimation and detection in traditional (centralized or decentralized, nonadaptive) inferential systems.

This work considers a diffusion network responding to streaming data, and studies the problem of identifying the topology of a subnetwork of observable agents by tracking their output measurements. Topology inference from indirect and/or incomplete datasets (network tomography) is in general an ill-posed problem. Under an appropriate Erdos-Renyi random graph model for the unobserved part, the problem of network tomography is well-posed in the thermodynamic limit: when the number of network agents grows to infinity, any arbitrary subnetwork topology associated with the observed agents can be recovered with high probability.

In this work, we consider multitask learning problems where clusters of nodes are interested in estimating their own parameter vector. Cooperation among clusters is beneficial when the optimal models of adjacent clusters have a good number of similar entries. We propose a fully distributed algorithm for solving this problem. The approach relies on minimizing a global mean-square error criterion regularized by nondifferentiable terms to promote cooperation among neighboring clusters. A general diffusion forward–backward splitting strategy is introduced. Then, it is specialized to the case of sparsity promoting regularizers. A closed-form expression for the proximal operator of a weighted sum of $\ell _1$ -norms is derived to achieve higher efficiency. We also provide conditions on the step-sizes that ensure convergence of the algorithm in the mean and mean-square error sense. Simulations are conducted to illustrate the effectiveness of the strategy.

We consider the problem of decentralized clustering and estimation over multi-task networks, where agents infer and track different models of interest. The agents do not know beforehand which model is generating their own data. They also do not know which agents in their neighborhood belong to the same cluster. We propose a decentralized clustering algorithm aimed at identifying and forming clusters of agents of similar objectives, and at guiding cooperation to enhance the inference performance. One key feature of the proposed technique is the integration of the learning and clustering tasks into a single strategy. We analyze the performance of the procedure and show that the error probabilities of types I and II decay exponentially to zero with the step-size parameter. While links between agents following different objectives are ignored in the clustering process, we nevertheless show how to exploit these links to relay critical information across the network for enhanced performance. Simulation results illustrate the performance of the proposed method in comparison to other useful techniques.

This work examines the mean-square error performance of diffusion stochastic algorithms under a generalized coordinate-descent scheme. In this setting, the adaptation step by each agent is limited to a random subset of the coordinates of its stochastic gradient vector. The selection of coordinates varies randomly from iteration to iteration and from agent to agent across the network. Such schemes are useful in reducing computational complexity at each iteration in power-intensive large data applications. They are also useful in modeling situations where some partial gradient information may be missing at random. Interestingly, the results show that the steady-state performance of the learning strategy is not always degraded, while the convergence rate suffers some degradation. The results provide yet another indication of the resilience and robustness of adaptive distributed strategies., Accepted for publication

Exploiting recent progress [1]-[4] in the characterization of the detection performance of diffusion strategies over adaptive multi-agent networks: i) we present two theoretical approximations, one based on asymptotic normality and the other based on the theory of exact asymptotics; and ii) we develop an efficient simulation method by tailoring the importance sampling technique to diffusion adaptation. We show that these theoretical and experimental tools complement each other well, with their combination offering a substantial advance for a reliable quantitative detection-performance assessment. The analysis provides insight into the interplay between the network topology, the combination weights, and the inference performance, revealing the universal behavior of diffusion-based detectors over adaptive networks.

This work studies distributed primal-dual strategies for adaptation and learning over networks from streaming data. Two first-order methods are considered based on the Arrow-Hurwicz (AH) and augmented Lagrangian (AL) techniques. Several revealing results are discovered in relation to the performance and stability of these strategies when employed over adaptive networks. The conclusions establish that the advantages that these methods have for deterministic optimization problems do not necessarily carry over to stochastic optimization problems. It is found that they have narrower stability ranges and worse steady-state mean-square-error performance than primal methods of the consensus and diffusion type. It is also found that the AH technique can become unstable under a partial observation model, while the other techniques are able to recover the unknown under this scenario. A method to enhance the performance of AL strategies is proposed by tying the selection of the step-size to their regularization parameter. It is shown that this method allows the AL algorithm to approach the performance of consensus and diffusion strategies but that it remains less stable than these other strategies., 16 pages, 9 figures

In this paper, we consider learning dictionary models over a network of agents, where each agent is only in charge of a portion of the dictionary elements. This formulation is relevant in Big Data scenarios where large dictionary models may be spread over different spatial locations and it is not feasible to aggregate all dictionaries in one location due to communication and privacy considerations. We first show that the dual function of the inference problem is an aggregation of individual cost functions associated with different agents, which can then be minimized efficiently by means of diffusion strategies. The collaborative inference step generates dual variables that are used by the agents to update their dictionaries without the need to share these dictionaries or even the coefficient models for the training data. This is a powerful property that leads to an effective distributed procedure for learning dictionaries over large networks (e.g., hundreds of agents in our experiments). Furthermore, the proposed learning strategy operates in an online manner and is able to respond to streaming data, where each data sample is presented to the network once., 16 pages, 8 figures. To appear in IEEE Transactions on Signal Processing

In this work, we study the task of distributed optimization over a network of learners in which each learner possesses a convex cost function, a set of affine equality constraints, and a set of convex inequality constraints. We propose a fully-distributed adaptive diffusion algorithm based on penalty methods that allows the network to cooperatively optimize the global cost function, which is defined as the sum of the individual costs over the network, subject to all constraints. We show that when small constant step-sizes are employed, the expected distance between the optimal solution vector and that obtained at each node in the network can be made arbitrarily small. Two distinguishing features of the proposed solution relative to other related approaches is that the developed strategy does not require the use of projections and is able to adapt to and track drifts in the location of the minimizer due to changes in the constraints or in the aggregate cost itself. The proposed strategy is also able to cope with changing network topology, is robust to network disruptions, and does not require global information or rely on central processors., 13 pages, 1 figure

Recent research works on distributed adaptive networks have intensively studied the case where the nodes estimate a common parameter vector collaboratively. However, there are many applications that are multitask-oriented in the sense that there are multiple parameter vectors that need to be inferred simultaneously. In this paper, we employ diffusion strategies to develop distributed algorithms that address clustered multitask problems by minimizing an appropriate mean-square error criterion with $\ell_2$-regularization. Some results on the mean-square stability and convergence of the algorithm are also provided. Simulations are conducted to illustrate the theoretical findings., 5 pages, 6 figures, submitted to ICASSP 2014

Background The tumor suppressor protein p53 plays important roles in DNA damage repair, cell cycle arrest and apoptosis. Due to its critical functions, the level of p53 is tightly regulated by a negative feedback mechanism to increase its tolerance towards fluctuations and disturbances. Interestingly, the p53 level is controlled by post-translational regulation rather than transcriptional regulation in this feedback mechanism. Results We analyzed the dynamics of this feedback to understand whether post-translational regulation provides any advantages over transcriptional regulation in regard to disturbance rejection. When a disturbance happens, even though negative feedback reduces the steady-state error, it can cause a system to become less stable and transiently overshoots, which may erroneously trigger downstream reactions. Therefore, the system needs to balance the trade-off between steady-state and transient errors. Feedback control and adaptive estimation theories revealed that post-translational regulation achieves a better trade-off than transcriptional regulation, contributing to a more steady level of p53 under the influence of noise and disturbances. Furthermore, post-translational regulation enables cells to respond more promptly to stress conditions with consistent amplitude. However, for better disturbance rejection, the p53- Mdm2 negative feedback has to pay a price of higher stochastic noise. Conclusions Our analyses suggest that the p53-Mdm2 feedback favors regulatory mechanisms that provide the optimal trade-offs for dynamic control.

In distributed processing, agents generally collect data generated by the same underlying unknown model (represented by a vector of parameters) and then solve an estimation or inference task cooperatively. In this paper, we consider the situation in which the data observed by the agents may have risen from two different models. Agents do not know beforehand which model accounts for their data and the data of their neighbors. The objective for the network is for all agents to reach agreement on which model to track and to estimate this model cooperatively. In these situations, where agents are subject to data from unknown different sources, conventional distributed estimation strategies would lead to biased estimates relative to any of the underlying models. We first show how to modify existing strategies to guarantee unbiasedness. We then develop a classification scheme for the agents to identify the models that generated the data, and propose a procedure by which the entire network can be made to converge towards the same model through a collaborative decision-making process. The resulting algorithm is applied to model fish foraging behavior in the presence of two food sources., 42 pages, 11 figures, to appear in IEEE Transactions on Signal Processing, 2014

The goal of this paper is to study the learning abilities of adaptive networks in the context of cognitive radio networks and to investigate how well they assist in allocating power and communications resources in the frequency domain. The allocation mechanism is based on a social foraging swarm model that lets every node allocate its resources (power/bits) in the frequency regions where the interference is at a minimum while avoiding collisions with other nodes. We employ adaptive diffusion techniques to estimate the interference profile in a cooperative manner and to guide the motion of the swarm individuals in the resource domain. A mean square performance analysis of the proposed strategy is provided and confirmed by simulation results. The proposed approach endows the cognitive network with powerful learning and adaptation capabilities, allowing fast reaction to dynamic changes in the spectrum. Numerical examples show how cooperative spectrum sensing remarkably improves the performance of the resource allocation technique based on swarming.

In this work we analyze the mean-square performance of different strategies for distributed estimation over least-mean-squares (LMS) adaptive networks. The results highlight some useful properties for distributed adaptation in comparison to fusion-based centralized solutions. The analysis establishes that, by optimizing over the combination weights, diffusion strategies can deliver lower excess-mean-square-error than centralized solutions employing traditional block or incremental LMS strategies. We first study in some detail the situation involving combinations of two adaptive agents and then extend the results to generic N-node ad-hoc networks. In the later case, we establish that, for sufficiently small step-sizes, diffusion strategies can outperform centralized block or incremental LMS strategies by optimizing over left-stochastic combination weighting matrices. The results suggest more efficient ways for organizing and processing data at fusion centers, and present useful adaptive strategies that are able to enhance performance when implemented in a distributed manner., 39 pages, 7 figures, to appear in IEEE Transactions on Signal Processing, 2012

Adaptive networks rely on in-network and collaborative processing among distributed agents to deliver enhanced performance in estimation and inference tasks. Information is exchanged among the nodes, usually over noisy links. The combination weights that are used by the nodes to fuse information from their neighbors play a critical role in influencing the adaptation and tracking abilities of the network. This paper first investigates the mean-square performance of general adaptive diffusion algorithms in the presence of various sources of imperfect information exchanges, quantization errors, and model non-stationarities. Among other results, the analysis reveals that link noise over the regression data modifies the dynamics of the network evolution in a distinct way, and leads to biased estimates in steady-state. The analysis also reveals how the network mean-square performance is dependent on the combination weights. We use these observations to show how the combination weights can be optimized and adapted. Simulation results illustrate the theoretical findings and match well with theory., 36 pages, 7 figures, to appear in IEEE Transactions on Signal Processing, June 2012

We study the problem of distributed least-squares estimation over ad hoc adaptive networks, where the nodes have a common objective to estimate and track a parameter vector. We consider the case where there is stationary additive colored noise on both the regressors and the output response, which results in biased local least-squares estimators. Assuming that the noise covariance can be estimated (or is known a priori), we first propose a bias-compensated recursive least-squares algorithm (BC-RLS). However, this bias compensation increases the variance or the mean-square deviation (MSD) of the local estimators, and errors in the noise covariance estimates may still result in residual bias. We demonstrate that the MSD and residual bias can then be significantly reduced by applying diffusion adaptation, i.e., by letting nodes combine their local estimates with those of their neighbors. We derive a necessary and sufficient condition for mean-square stability of the algorithm, under some mild assumptions. Furthermore, we derive closed-form expressions for its steady-state mean and mean-square performance. Simulation results are provided, which agree well with the theoretical results. We also consider some special cases where the mean-square stability improvement of diffusion BC-RLS over BC-RLS can be mathematically verified. © 2011 IEEE. ispartof: IEEE Transactions on Signal Processing vol:59 issue:11 pages:5212-5224 status: published

Coherent optical links enable high-density constellations and, consequently, a higher throughput. However, the phase noise associated with the transmitter and the receiver lasers is a challenging issue in coherent lightwave systems. The authors present an approach that relies on using digital signal processing techniques to compensate for the laser phase-noise effects. The proposed approach exploits the digital processing power to address the problems arising from optical imperfections. The authors present an adaptive filtering scheme that reduces the effect of the laser phase noise and, consequently, relaxes the laser linewidth requirement. The proposed approach shows how the signal processing techniques can be exploited to compensate for the optical impairments by utilizing the continuing scale down in size and power in very large scale integration (VLSI) technology.

Kalman filtering has been proposed in the literature for wireless channel estimation, however, it is not sufficiently robust to uncertainties in the channel auto-correlation model as well as to multiple access interference (MAI). This paper presents a receiver structure for direct-sequence code-division multiple-access (DS-CDMA) systems by using robust Kalman estimation and post-correlation (i.e., symbol rate) processing for channel estimation. The proposed structure is also generalized to incorporate multiple-antenna combining and interference cancellation techniques. The resulting receiver outperforms earlier structures in the presence of channel modeling uncertainties, MAI, and low-received signal-to-noise ratio. The enhancement in performance is achieved at the same order of complexity as a standard Kalman-based receiver.

This paper develops a unified approach to the analysis and design of adaptive filters with error nonlinearities. In particular, the paper performs stability and steady-state analysis of this class of filters under weaker conditions than what is usually encountered in the literature, and without imposing any restriction on the color or statistics of the input. The analysis results are subsequently used to derive an expression for the optimum nonlinearity, which turns out to be a function of the probability density function of the estimation error. Some common nonlinearities are shown to be approximations to the optimum nonlinearity. The framework pursued here is based on energy conservation arguments.