Search

In this paper, two different timing-mismatch compensation strategies for two-channel time-interleaved analog-to-digital converters are comprehensively analyzed and compared. In the first strategy (SA), one channel serves as a reference channel, and the other channel is compensated to match the reference channel using the timing mismatch between the channels. In the second strategy (SB), both channels are compensated to match each other, using half the value of the channel mismatch with different signs. For SB, the paper introduces a novel compensation structure that utilizes parallel differentiator-multiplier (PDM) branches. Expressions for the spurious-free dynamic range (SFDR) after compensation are derived for both strategies. These expressions reveal that the novel PDM structure achieves a remarkably greater SFDR than the existing cascaded differentiator-multiplier (CDM) compensation structure which can be used for both SA and SB. This is because, after compensation, the remaining aliasing distortion in the proposed PDM structure is shown to be of higher approximation order (in terms of the mismatch value) and thus substantially smaller than in the CDM structure. Simulations included in the paper validate the theoretical results. [ABSTRACT FROM AUTHOR]

Real-world phenomena that can be formulated as signals are often affected by a number of factors and appear as multi-component modes. To understand and process such phenomena, “divide-and-conquer” is probably the most common strategy to address the problem. In other words, the captured signal is decomposed into signal components for each individual component to be processed. Unfortunately, for signals that are superimposition of non-stationary amplitude-frequency modulated (AM-FM) components, the “divide-and-conquer” strategy is bound to fail, since there is no way to be sure that the decomposed components take on the AM-FM formulations which are necessary for the extraction of their instantaneous frequencies (IFs) and amplitudes (IAs). In this paper, we propose an adaptive signal separation operation (ASSO) for effective and accurate separation of a single-channel blind-source multi-component signal, via introducing a time-varying parameter that adapts locally to IFs and using linear chirp (linear frequency modulation) signals to approximate components at each time instant. We derive more accurate component recovery formulae based on the linear chirp signal local approximation. In addition, a recovery scheme, together with a ridge detection method, is also proposed to extract the signal components one by one, and the time-varying parameter is updated for each component. The proposed method is suitable for engineering implementation, being capable of separating complicated signals into their components or sub-signals and reconstructing the signal trend directly. Numerical experiments on synthetic and real-world signals are presented to demonstrate our improvement over the previous attempts. [ABSTRACT FROM AUTHOR]

This paper studies the robust matrix completion (RMC) problem with the objective to recover a low-rank matrix from partial observations that may contain significant errors. If all the observations in one column are erroneous, existing RMC methods can locate the corrupted column at best but cannot recover the actual data in that column. Low-rank Hankel matrices characterize the additional correlations among columns besides the low-rankness and exist in power system monitoring, magnetic resonance imaging (MRI) imaging, and array signal processing. Exploiting the low-rank Hankel property, this paper develops an alternating-projection-based fast algorithm to solve the nonconvex RMC problem. The algorithm converges to the ground-truth low-rank matrix with a linear rate even when all the measurements in a constant fraction of columns are corrupted. The required number of observations is significantly less than the existing bounds for the conventional RMC. Numerical results are reported to evaluate the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes a novel algorithm to determine the optimal orientation of sensing axes of redundant inertial sensors such as accelerometers and gyroscopes (gyros) for increasing the sensing accuracy. In this paper, we have proposed a novel iterative algorithm to find the optimal sensor configuration. The proposed algorithm utilizes the majorization-minimization (MM) algorithm and the duality principle to find the optimal configuration. Unlike the state-of-the-art approaches which are mainly geometrical in nature and restricted to sensors’ noise being uncorrelated, the proposed algorithm gives the exact orientations of the sensors and can easily deal with the cases of correlated noise. The proposed algorithm has been implemented and tested via numerical simulation in the MATLAB. The simulation results show that the algorithm converges to the optimal configurations and shows the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes a momentum algorithm based on Borges difference and an adaptive momentum (Adam) algorithm based on Borges difference to update parameters, which can adjust the momentum information more flexibly. The Borges difference is proposed from the definition of Borges derivative to be combined with the gradient algorithm in convolutional neural networks. The proposed momentum algorithm based on Borges difference and Adam algorithm based on Borges difference can be adjusted more flexibly in order to speed up the convergence. The parameter optimization algorithm with the Borges difference presents a better performance compared with the integer-order momentum algorithm and integer-order Adam algorithm, with the proposed nonlinear adjustment method for the parameter tuning of convolutional neural networks. By analyzing experimental results of Fashion-MNIST dataset and CIFAR-10 dataset, the optimization algorithms based on Borges difference proposed in this paper gain better effects on the optimization model compared with the corresponding ones based on the integer-order difference, and can speed up the convergence speed and recognition accuracy of the image recognition. [ABSTRACT FROM AUTHOR]

Distributed or decentralized estimation of covariance, and distributed principal component analysis have been introduced and studied in the signal processing community in recent years, and applications in array processing have been indicated in some detail. Inspired by these, this paper provides a detailed development of several distributed algorithms for array processing. New distributed algorithms are proposed for DOA estimation methods like root-MUSIC, total least squares-ESPRIT, and FOCUSS. Other contributions include distributed design of the Capon beamformer from data, and distributed implementation of the spatial smoothing method for coherent sources. A distributed implementation of a recently proposed beamspace method called the convolutional beamspace (CBS) is also proposed. The proposed algorithms are fully distributed – an average consensus (AC) is used to avoid the need for a fusion center. The algorithms are based on a recently reported finite-time version of AC which converges to the exact solution in a finite number of iterations. Numerical examples are given throughout the paper to show the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Over the decades, multiple approaches have been proposed to solve convex programs. The development of interior-point methods allowed solving a more general set of convex programs known as semi-definite and second-order cone programs. However, these methods are excessively slow for high dimensions. On the other hand, optimization algorithms on manifolds have shown great abilities in finding solutions to non-convex problems in a reasonable time. This paper suggests using a Riemannian optimization approach to solve a subset of convex optimization problems wherein the optimization variable is a doubly stochastic matrix. Optimization over the set of doubly stochastic matrices is crucial for multiple communications and signal processing applications, especially graph-based clustering. The paper introduces and investigates the geometries of three convex manifolds, namely the doubly stochastic, the symmetric, and the definite multinomial manifolds which generalize the simplex, also known as the multinomial manifold. Theoretical complexity analysis and numerical simulation results testify the efficiency of the proposed method over state-of-the-art algorithms. In particular, they reveal that the proposed framework outperforms conventional generic and specialized approaches, especially in high dimensions. [ABSTRACT FROM AUTHOR]

In this paper, we propose new methodologies to decrease the computational cost of safe screening tests for LASSO. We first introduce a new screening strategy, dubbed “joint screening test,” which allows the rejection of a set of atoms by performing one single test. Our approach enables to find good compromises between complexity of implementation and effectiveness of screening. Second, we propose two new methods to decrease the computational cost inherent to the construction of the (so-called) “safe region.” Our numerical experiments show that the proposed procedures lead to significant computational gains as compared to standard methodologies. [ABSTRACT FROM AUTHOR]

This paper introduces a node-asynchronous communication protocol in which an agent in a network wakes up randomly and independently, collects states of its neighbors, updates its own state, and then broadcasts back to its neighbors. This protocol differs from consensus algorithms and it allows distributed computation of an arbitrary eigenvector of the network, in which communication between agents is allowed to be directed. (The graph operator is still required to be a normal matrix). To analyze the scheme, this paper studies a random asynchronous variant of the power iteration. Under this random asynchronous model, an initial signal is proven to converge to an eigenvector of eigenvalue 1 (a fixed point) even in the case of operator having spectral radius larger than unity. The rate of convergence is shown to depend not only on the eigenvalue gap but also on the eigenspace geometry of the operator as well as the amount of asynchronicity of the updates. In particular, the convergence region for the eigenvalues gets larger as the updates get less synchronous. Random asynchronous updates are also interpreted from the graph signal perspective, and it is shown that a non-smooth signal converges to the smoothest signal under the random model. When the eigenvalues are real, second order polynomials are used to achieve convergence to an arbitrary eigenvector of the operator. Using second order polynomials the paper formalizes the node-asynchronous communication model. As an application, the protocol is used to compute the Fiedler vector of a network to achieve autonomous clustering. [ABSTRACT FROM AUTHOR]

In this paper, we devise a communication-efficient decentralized algorithm, named as communication-censored alternating direction method of multipliers (ADMM) (COCA), to solve a convex consensus optimization problem defined over a network. Similar to popular decentralized consensus optimization algorithms such as ADMM, at every iteration of COCA, a node exchanges its local variable with neighbors, and then updates its local variable according to the received neighboring variables and its local cost function. A different feature of COCA is that a node is not allowed to transmit its local variable to neighbors, if this variable is not sufficiently different to the previously transmitted one. The sufficiency of the difference is evaluated by a properly designed censoring function. Though this censoring strategy may slow down the optimization process, it effectively reduces the communication cost. We prove that when the censoring function is properly chosen, COCA converges to an optimal solution of the convex consensus optimization problem. Furthermore, if the local cost functions are strongly convex, COCA has a fast linear convergence rate. Numerical experiments demonstrate that, given a target solution accuracy, COCA is able to significantly reduce the overall communication cost compared to existing algorithms including ADMM, and hence fits for applications where network communication is a bottleneck. [ABSTRACT FROM AUTHOR]

This paper proposes a new framework for the design of transmit and receive beamformers for interference alignment (IA) without symbol extensions in multi-antenna cellular networks. We consider IA in a G cell network with K users/cell, N antennas at each base station (BS) and M antennas at each user. The proposed framework is developed by recasting the conditions for IA as two sets of rank constraints, one on the rank of interference matrices, and the other on the transmit beamformers in the uplink. The interference matrix consists of all the interfering vectors received at a BS from the out-of-cell users in the uplink. Using these conditions and the crucial observation that the rank of interference matrices under alignment can be determined beforehand, this paper develops two sets of algorithms for IA. The first part of this paper develops rank minimization algorithms for IA by iteratively minimizing a weighted matrix norm of the interference matrix. Different choices of matrix norms lead to reweighted nuclear norm minimization (RNNM) or reweighted Frobenius norm minimization (RFNM) algorithms with significantly different per-iteration complexities. Alternately, the second part of this paper devises an alternating minimization (AM) algorithm where the rank-deficient interference matrices are expressed as a product of two lower-dimensional matrices that are then alternately optimized. Simulation results indicate that RNNM, which has a per-iteration complexity of a semidefinite program, is effective in designing aligned beamformers for proper-feasible systems with or without redundant antennas, while RFNM and AM, which have a per-iteration complexity of a quadratic program, are better suited for systems with redundant antennas. [ABSTRACT FROM PUBLISHER]

The present paper elaborates on tangent-bundle maps on the Grassmann manifold, with application to subspace arithmetic averaging. In particular, the present contribution elaborates on the work about retraction/lifting maps devised for the Stiefel manifold in the recently published paper T. Kaneko, S. Fiori and T. Tanaka, “Empirical arithmetic averaging over the compact Stiefel manifold,” IEEE Trans. Signal Process., Vol. 61, No. 4, pp. 883–894, February 2013, and discusses the extension of such maps to the Grassmann manifold. Tangent-bundle maps are devised on the basis of the thin QR matrix decomposition, the polar matrix decomposition and the exponential map. Also, tangent-bundle pseudo-maps based on the matrix Cayley transform are devised. Theoretical and numerical comparisons about the devised tangent-bundle maps are performed in order to get an insight into their relative merits and demerits, with special emphasis to their computational burden. The averaging algorithm based on the thin-QR decomposition maps stands out as it exhibits the best trade off between numerical precision and computational burden. Such algorithm is further compared with two Grassmann averaging algorithms drawn from the scientific literature on an handwritten digits recognition data set. The thin-QR tangent-bundle maps-based algorithm exhibits again numerical features that make it preferable over such algorithms. [ABSTRACT FROM PUBLISHER]

This paper considers an Internet-of-Things (IoT) scenario in which devices sporadically transmit short packets with few pilot symbols over a fading channel. Devices are characterized by unique transmission non-idealities, such as I/Q imbalance. The number of pilots is generally insufficient to obtain an accurate estimate of the end-to-end channel, which includes the effects of fading and of the transmission-side distortion. This paper proposes to tackle this problem by using meta-learning. Accordingly, pilots from previous IoT transmissions are used as meta-training data in order to train a demodulator that is able to quickly adapt to new end-to-end channel conditions from few pilots. Various state-of-the-art meta-learning schemes are adapted to the problem at hand and evaluated, including Model-Agnostic Meta-Learning (MAML), First-Order MAML (FOMAML), REPTILE, and fast Context Adaptation VIA meta-learning (CAVIA). Both offline and online solutions are developed. In the latter case, an integrated online meta-learning and adaptive pilot number selection scheme is proposed. Numerical results validate the advantages of meta-learning as compared to training schemes that either do not leverage prior transmissions or apply a standard joint learning algorithms on previously received data. [ABSTRACT FROM AUTHOR]

Signal classification problems arise in a wide variety of applications, and their demand is only expected to grow. In this paper, we focus on the wireless sensor network signal classification setting, where each sensor forwards quantized signals to a fusion center to be classified. Our primary goal is to train a decision function and quantizers across the sensors to maximize the classification performance in an online manner. Moreover, we are interested in sparse sensor selection using a marginalized weighted kernel approach to improve network resource efficiency by disabling less reliable sensors with minimal effect on classification performance. To achieve our goals, we develop a multi-sensor online kernel scalar quantization (MSOKSQ) learning strategy that operates on the sensor outputs at the fusion center. Our theoretical analysis reveals how the proposed algorithm affects the quantizers across the sensors. Additionally, we provide a convergence analysis of our online learning approach by studying its relationship to batch learning. We conduct numerical studies under different classification and sensor network settings which demonstrate the accuracy gains from optimizing different components of MSOKSQ and robustness to reduction in the number of sensors selected. [ABSTRACT FROM AUTHOR]

This paper introduces Distributed Stein Variational Gradient Descent (DSVGD), a non-parametric generalized Bayesian inference framework for federated learning. DSVGD maintains a number of non-random and interacting particles at a central server to represent the current iterate of the model global posterior. The particles are iteratively downloaded and updated by a subset of agents with the end goal of minimizing the global free energy. By varying the number of particles, DSVGD enables a flexible trade-off between per-iteration communication load and number of communication rounds. DSVGD is shown to compare favorably to benchmark frequentist and Bayesian federated learning strategies in terms of accuracy and scalability with respect to the number of agents, while also providing well-calibrated, and hence trustworthy, predictions. [ABSTRACT FROM AUTHOR]

This paper considers the problem of cooperative localization in mobile networks. In static networks, node locations can be obtained from pairwise distance measurements using the classical multidimensional scaling (MDS) approach. This paper introduces a modified MDS framework that also incorporates relative velocity measurements if available in mobile networks. The proposed cost function is minimized via a low complexity majorization algorithm. A distributed variant of the proposed algorithm is also outlined for use in large resource-constrained multi-hop networks. The proposed algorithm incurs low computational and communication cost, and handles practical constraints such as missing measurements and variable node velocities. Simulation results corroborate the performance gains obtained by the proposed algorithm over state-of-the-art localization algorithms. [ABSTRACT FROM AUTHOR]

Downsampling and the computation of the bandlimited interpolation of discrete-time signals are two important concepts in signal processing. In this paper we analyze the downsampling operation regarding its impact on the existence and computability of the bounded bandlimited interpolation. We assume that the discrete-time signal is obtained by downsampling the samples of a bounded bandlimited signal that vanishes at infinity, and we study two problems. First, we investigate the existence of the bounded bandlimited interpolation for such discrete-time signals from a signal theoretic perspective and show that there exist signals for which the bounded bandlimited interpolation does not exist. Second, we analyze the algorithmic generation of the bounded bandlimited interpolation, using the concept of Turing computability. Turing computability models what is theoretically implementable on a digital computer. Interestingly, it turns out that even if the bounded bandlimited interpolation exists analytically, it is not always computable, which implies that there exists no algorithm on a digital computer that can always compute it. Computability is important in order that the approximation error be controlled. If a signal is not computable, we cannot ascertain whether the computed signal is sufficiently close to the true signal, i.e., we cannot verify every approximation accuracy. [ABSTRACT FROM AUTHOR]

This two-part paper considers an uplink massive device communication scenario in which a large number of devices are connected to a base station (BS), but user traffic is sporadic so that in any given coherence interval, only a subset of users is active. The objective is to quantify the cost of active user detection and channel estimation and to characterize the overall achievable rate of a grant-free two-phase access scheme in which device activity detection and channel estimation are performed jointly using pilot sequences in the first phase and data is transmitted in the second phase. In order to accommodate a large number of simultaneously transmitting devices, this paper studies an asymptotic regime where the BS is equipped with a massive number of antennas. The main contributions of Part I of this paper are as follows. First, we note that as a consequence of having a large pool of potentially active devices but limited coherence time, the pilot sequences cannot all be orthogonal. However, despite the nonorthogonality, this paper shows that in the asymptotic massive multiple-input multiple-output regime, both the missed device detection and the false alarm probabilities for activity detection can always be made to go to zero by utilizing compressed sensing techniques that exploit sparsity in the user activity pattern. Part II of this paper further characterizes the achievable rates using the proposed scheme and quantifies the cost of using nonorthogonal pilot sequences for channel estimation in achievable rates. [ABSTRACT FROM AUTHOR]

Following the Unlimited Sampling strategy to alleviate the omnipresent dynamic range barrier, we study the problem of recovering a bandlimited signal from point-wise modulo samples, aiming to connect theoretical guarantees with hardware implementation considerations. Our starting point is a class of non-idealities that we observe in prototyping an unlimited sampling based analog-to-digital converter. To address these non-idealities, we provide a new Fourier domain recovery algorithm. Our approach is validated both in theory and via extensive experiments on our prototype analog-to-digital converter, providing the first demonstration of unlimited sampling for data arising from real hardware, both for the current and previous approaches. Advantages of our algorithm include that it is agnostic to the modulo threshold and it can handle arbitrary folding times. We expect that the end-to-end realization studied in this paper will pave the path for exploring the unlimited sampling methodology in a number of real world applications. [ABSTRACT FROM AUTHOR]

The problem of range estimation based on noisy measurements with phase ambiguity can be solved using signals across multiple frequencies by existing algorithms based on either the Chinese Remainder Theorem or lattice theory. The performances of these algorithms are constrained by the trade-off between SNR of measurements and required estimation accuracy. In this paper, we firstly propose an algorithm to find a set of wavelengths satisfying the conditions to apply the closed-form lattice algorithm from a given interval. Then we propose an enhanced lattice algorithm for range estimation based on existing lattice approaches. In the conventional lattice algorithm, the selection of an integer vector by the estimator often fails to be the ground truth because of noise. As a result, the true value can be a neighbouring lattice point of the estimated point where neighbours are defined in terms of a distance depending on the noise characteristics. In this paper, all neighbouring lattice points of the estimated point obtained by a conventional lattice algorithm are regarded as candidates for the new estimate. Then an efficient estimator is presented to choose one of those candidates to be the new estimate. The performance of the proposed algorithm, i.e. the successful probability of reconstruction, is derived mathematically. Additionally, a computation-reduced algorithm is presented to reduce the overhead of computation. The calculated probability and simulation results demonstrate the efficiency of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes a low complexity n-dimensional (nD) FastICA algorithm and architecture by introducing the concept of coordinate rotation where n\geq 2. The proposed algorithm can merge the two key steps of conventional FastICA—preprocessing and update and is therefore capable of reducing the hardware complexity of the conventional FastICA significantly as demonstrated in this paper. Hardware implementation can further be simplified due to the recursive nature of the proposed algorithm where the same 2D hardware module can be used as the fundamental core to implement nD architecture. Together with the algorithm formulation, its functionality is also validated and hardware complexity is analyzed and compared with the conventional nD FastICA. [ABSTRACT FROM PUBLISHER]

The Unlimited Sensing Framework (USF) was recently introduced to overcome the sensor saturation bottleneck in conventional digital acquisition systems. At its core, the USF converts a continuous-time high-dynamic-range (HDR) signal into folded, low-dynamic-range, modulo samples and allows the recovery of the HDR signal via algorithmic unfolding. In hardware, however, implementing ideal modulo folding requires careful calibration, analog design and high precision. At the interface of theory and practice, this paper explores a computational sampling strategy that relaxes strict hardware requirements via a novel, mathematically guaranteed reconstruction. We start with a generalized model for USF with two new parameters modeling hysteresis and folding transients in addition to the modulo threshold. Hysteresis accounts for mismatches between the reset threshold and the amplitude displacement at the folding time and the folding transient is a continuous transition period in the implementation of a reset. Both these effects are motivated by our hardware experiments and also occur in previous, domain-specific applications. We show that hysteresis is beneficial for USF and leverage it to derive the first recovery guarantees in the context of our generalized USF model for a certain sampling rate regime. Additionally, we show how the sampling rate requirement can be greatly reduced via a direct generalization of the proposed recovery. Our theoretical work is corroborated by hardware experiments with a hysteresis enabled, modulo ADC testbed comprising off-the-shelf electronic components. Thus, by capitalizing on a collaboration between hardware and algorithms, our paper enables an end-to-end pipeline for HDR sampling allowing more flexible hardware implementations. [ABSTRACT FROM AUTHOR]

In this paper, we propose to extend the standard Convolutional Dictionary Learning problem to a tensor representation where the activations are constrained to be “low-rank” through a Canonical Polyadic decomposition. We show that this additional constraint increases the robustness of the CDL with respect to noise and improve the interpretability of the final results. In addition, we discuss in detail the advantages of this representation and introduce two algorithms, based on ADMM or FISTA, that efficiently solve this problem. We show that by exploiting the low rank property of activations, they achieve lower complexity than the main CDL algorithms. Finally, we evaluate our approach on a wide range of experiments, highlighting the modularity and the advantages of this tensorial low-rank formulation. [ABSTRACT FROM AUTHOR]

Distributed primal-dual methods have been widely used for solving large-scale constrained optimization problems. The majority of existing results focus on the problems with decoupled constraints. Some recent works have studied the problems subject to separable globally coupled constraints. This paper considers the distributed optimization problems with globally coupled constraints over networks without requiring the separability of the globally coupled constraints. This is made possible by the local estimates of the constraint violations. For solving such a problem, we propose a primal-dual algorithm in the augmented Lagrangian framework, combining the average consensus technique. We first establish a non-ergodic convergence rate of $\mathcal {O}(1/k)$ in terms of the objective residual for solving a distributed constrained convex optimization problem, where $k$ is the iteration counter. Specifically, the global objective function is the aggregate of the local convex and possibly non-smooth costs, and the coupled constraint is the sum of the local linear equality constraints. The numerical results illustrate the performance of the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a decentralized algorithm for solving a consensus optimization problem defined in a static networked multi-agent system, where the local objective functions have the smooth+nonsmooth composite form. Examples of such problems include decentralized constrained quadratic programming and compressed sensing problems, as well as many regularization problems arising in inverse problems, signal processing, and machine learning, which have decentralized applications. This paper addresses the need for efficient decentralized algorithms that take advantages of proximal operations for the nonsmooth terms. We propose a proximal gradient exact first-order algorithm (PG-EXTRA) that utilizes the composite structure and has the best known convergence rate. It is a nontrivial extension to the recent algorithm EXTRA. At each iteration, each agent locally computes a gradient of the smooth part of its objective and a proximal map of the nonsmooth part, as well as exchanges information with its neighbors. The algorithm is “exact” in the sense that an exact consensus minimizer can be obtained with a fixed step size, whereas most previous methods must use diminishing step sizes. When the smooth part has Lipschitz gradients, PG-EXTRA has an ergodic convergence rate of O\left(1\over k\right) in terms of the first-order optimality residual. When the smooth part vanishes, PG-EXTRA reduces to P-EXTRA, an algorithm without the gradients (so no “G” in the name), which has a slightly improved convergence rate at o\left(1\over k\right) in a standard (non-ergodic) sense. Numerical experiments demonstrate effectiveness of PG-EXTRA and validate our convergence results [ABSTRACT FROM PUBLISHER]

Extracting latent low-dimensional structure from high-dimensional data is of paramount importance in timely inference tasks encountered with “Big Data” analytics. However, increasingly noisy, heterogeneous, and incomplete datasets, as well as the need for real-time processing of streaming data, pose major challenges to this end. In this context, the present paper permeates benefits from rank minimization to scalable imputation of missing data, via tracking low-dimensional subspaces and unraveling latent (possibly multi-way) structure from incomplete streaming data. For low-rank matrix data, a subspace estimator is proposed based on an exponentially weighted least-squares criterion regularized with the nuclear norm. After recasting the nonseparable nuclear norm into a form amenable to online optimization, real-time algorithms with complementary strengths are developed, and their convergence is established under simplifying technical assumptions. In a stationary setting, the asymptotic estimates obtained offer the well-documented performance guarantees of the batch nuclear-norm regularized estimator. Under the same unifying framework, a novel online (adaptive) algorithm is developed to obtain multi-way decompositions of low-rank tensors with missing entries and perform imputation as a byproduct. Simulated tests with both synthetic as well as real Internet and cardiac magnetic resonance imagery (MRI) data confirm the efficacy of the proposed algorithms, and their superior performance relative to state-of-the-art alternatives. [ABSTRACT FROM PUBLISHER]

Recently, it has been shown that the stability of the least mean fourth (LMF) algorithm depends on the nonstationarity of the plant. The present paper investigates the possibility of overcoming this problem by normalization of the weight vector update term. A rigorous mean-square stability analysis is provided for a recent normalized LMF algorithm, which is normalized by a term that is second order in the estimation error and fourth order in the regressor. The analysis is done for a Markov plant with a stationary white input with even probability density function and a stationary zero-mean white noise. It is proved that the mean-square deviation (MSD) of the algorithm is bounded for all finite values of the input variance, noise variance, initial MSD, and mean-square plant parameter increment. Analytical results are supported by simulations. [ABSTRACT FROM PUBLISHER]

This paper concerns the sparse Bayesian learning (SBL) problem for group sparse signals. Group sparsity means that the signal coefficients can be divided into groups and that the entries in one group are simultaneously zero or nonzero. In SBL, each group is controlled by a hyperparameter, which is estimated by solving the marginal likelihood maximization (MLM) problem. MLM is used to maximize the marginal likelihood of a given hyper-parameter by fixing all others. The main contribution of this paper is to extend the fast marginal likelihood maximization (FMLM) method to the group sparsity case. The key process of realizing FMLM in the group sparsity case is solved by finding the roots of a polynomial. Hence, the local maximum of the marginal likelihood can be found exactly, and the final estimation is obtained more efficiently. Furthermore, most large matrix inverses involved in MLM are replaced with the eigenvalue decompositions of much smaller matrices, which substantially reduces the computational complexity. Numerical results with both simulated and real data demonstrate the excellent performance and relatively high computational efficiency of the proposed method. [ABSTRACT FROM PUBLISHER]

This paper deals with change detection of utility maximization behavior given a dataset. In online social media, such changes occur due to the effect of marketing, advertising, or changes in ground truth. First, we use the revealed preference framework to detect the unknown time point (change point) at which the utility function changed. We derive necessary and sufficient conditions for detecting the change point. Second, in the presence of noisy measurements, we propose a method to detect the change point and construct a decision test. Also, an optimization criteria is provided to recover the linear perturbation coefficients. Finally, to reduce the computational cost, a dimensionality reduction algorithm using Johnson–Lindenstrauss transform is presented. The results developed are illustrated on two real datasets: Yahoo! Tech Buzz dataset and Youstatanalyzer dataset. By using the results developed in the paper, several useful insights can be gleaned from these datasets. First, the changes in ground truth affecting the utility of the agent can be detected by utility maximization behavior in online search. Second, the recovered utility functions satisfy the single crossing property indicating strategic substitute behavior in online search. Finally, due to the large number of videos in YouTube, the utility maximization behavior was verified through the dimensionality reduction algorithm. [ABSTRACT FROM PUBLISHER]

This paper studies the convergence properties of the well-known message-passing algorithm for convex optimisation. Under the assumption of pairwise separability and scaled diagonal dominance, asymptotic convergence is established and a simple bound for the convergence rate is provided for message-passing. In comparison with previous results, our results do not require the given convex program to have known convex pairwise components and that our bound for the convergence rate is tighter and simpler. When specialised to quadratic optimisation, we generalise known results by providing a very simple bound for the convergence rate. [ABSTRACT FROM AUTHOR]

We consider the problem of minimizing the $\ell _1$ norm of a linear map over the sphere, which arises in various machine learning applications such as orthogonal dictionary learning (ODL) and robust subspace recovery (RSR). The problem is numerically challenging due to its nonsmooth objective and nonconvex constraint, and its algorithmic aspects have not been well explored. In this paper, we show how the manifold structure of the sphere can be exploited to design fast algorithms with provable guarantees for tackling this problem. Specifically, our contribution is fourfold. First, we present a manifold proximal point algorithm (ManPPA) for the problem and show that it converges at a global sublinear rate. Furthermore, we show that ManPPA can achieve a local quadratic convergence rate when applied to sharp instances of the problem. Second, we develop a semismooth Newton-based inexact augmented Lagrangian method for computing the search direction in each iteration of ManPPA and show that it has an asymptotic superlinear convergence rate. Third, we propose a stochastic variant of ManPPA called StManPPA, which is well suited for large-scale computation, and establish its sublinear convergence rate. Both ManPPA and StManPPA have provably faster convergence rates than existing subgradient-type methods. Fourth, using ManPPA as a building block, we propose a new heuristic method for solving a matrix analog of the problem, in which the sphere is replaced by the Stiefel manifold. The results from our extensive numerical experiments on the ODL and RSR problems demonstrate the efficiency and efficacy of our proposed methods. [ABSTRACT FROM AUTHOR]

Stochastic compositional optimization generalizes classic (non-compositional) stochastic optimization to the minimization of compositions of functions. Each composition may introduce an additional expectation. The series of expectations may be nested. Stochastic compositional optimization is gaining popularity in applications such as reinforcement learning and meta learning. This paper presents a new Stochastically Corrected Stochastic Compositional gradient method (SCSC). SCSC runs in a single-time scale with a single loop, uses a fixed batch size, and guarantees to converge at the same rate as the stochastic gradient descent (SGD) method for non-compositional stochastic optimization. This is achieved by making a careful improvement to a popular stochastic compositional gradient method. It is easy to apply SGD-improvement techniques to accelerate SCSC. This helps SCSC achieve state-of-the-art performance for stochastic compositional optimization. In particular, we apply Adam to SCSC, and the exhibited rate of convergence matches that of the original Adam on non-compositional stochastic optimization. We test SCSC using the model-agnostic meta-learning tasks. [ABSTRACT FROM AUTHOR]

The conventional model of disjunctive group testing assumes that there are several defective elements (or defectives) among a large population, and a group test yields the positive response if and only if the testing group contains at least one defective element. The basic problem is to find all defectives using a minimal possible number of group tests. However, when the number of defectives is unknown there arises an additional problem, namely: how to estimate the random number of defective elements. In this paper, we concentrate on testing the hypothesis $H_0$ : the number of defectives $\leq s_1$ against the alternative hypothesis $H_1$ : the number of defectives $\geq s_2$. We introduce a new decoding algorithm based on the comparison of the number of tests having positive responses with an appropriate fixed threshold. For some asymptotic regimes on $s_1$ and $s_2$ , the proposed algorithm is shown to be order-optimal. Additionally, our simulation results verify the advantages of the proposed algorithm such as low complexity and a small error probability compared with known algorithms. [ABSTRACT FROM AUTHOR]

Secret key generation refers to the problem of generating a common secret key without revealing any information about it to an eavesdropper. All users observe correlated components of a common source and can further use a noisy public channel for discussion, which is open to eavesdroppers. A related problem is that of secure authentication, which has structural similarities and connections to the first problem. For authentication, users need to be enrolled and securely authenticated while minimizing the privacy leakage rate. This paper studies the algorithmic computability of the forward secret key capacity and the secure authentication capacity. For the algorithmic computability, the concept of a Turing machine is used as it provides fundamental performance limits for today's digital computers. In this paper, it is shown that the forward secret key capacity with a noisy public channel is not computable and consequently, there is no algorithm that can simulate or compute the secret key capacity; even if there are no limitations on computational complexity and computing power. On the other hand, if the public channel is noiseless so that there are no rate constraints on the public communication, the secret key capacity is a computable continuous function, which is the strongest form of computability. A similar behavior is subsequently observed for the authentication problem: The secure authentication capacity under storage rate and privacy leakage rate constraints is not computable, while the case without privacy leakage rate constraints is computable. [ABSTRACT FROM AUTHOR]

This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and nonsmooth terms. Specifically, the smooth and nonsmooth terms are dealt with by gradient and proximal updates, respectively. The proposed algorithm is closely related to a previous algorithm, PG-EXTRA (W. Shi, Q. Ling, G. Wu, and W. Yin, “A proximal gradient algorithm for decentralized composite optimization,” IEEE Trans. Signal Process., vol. 63, no. 22, pp. 6013–6023, 2015), but has a few advantages. First of all, agents use uncoordinated step-sizes, and the stable upper bounds on step-sizes are independent of network topologies. The step-sizes depend on local objective functions, and they can be as large as those of the gradient descent. Second, for the special case without nonsmooth terms, linear convergence can be achieved under the strong convexity assumption. The dependence of the convergence rate on the objective functions and the network are separated, and the convergence rate of the new algorithm is as good as one of the two convergence rates that match the typical rates for the general gradient descent and the consensus averaging. We provide numerical experiments to demonstrate the efficacy of the introduced algorithm and validate our theoretical discoveries. [ABSTRACT FROM AUTHOR]

The paper derives the stability bound of the initial mean-square deviation of an adaptive filtering algorithm based on minimizing the 2Lth moment of the estimation error, with L being an integer greater than 1. The analysis is done for a time-invariant plant with even input probability density function. Dependence of the stability bound on the algorithm step-size, type of the noise distribution, signal-to-noise ratio (SNR), and L is studied. It is shown that the stability bound is decreasing in the step-size. The stability bound decreases as the tail of the noise probability density function becomes lighter for the same steady-state misadjustment. This result is surprising since, for example, it is expected that the stability of the least mean 2Lth algorithm for binary noise distribution is better than that for uniform distribution. The stability bound is proportional to the reciprocal of the SNR, another surprising result. Finally, the stability bound is decreasing in L, which implies a limitation on the ability to improve the performance of the adaptive filter by increasing the order of the cost function. Theoretical results are supported by simulations. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new algorithm for sequential non-parametric hypothesis testing based on Random Distortion Testing (RDT). The data-based approach is non-parametric in the sense that the underlying signal distributions under each hypothesis are assumed to be unknown. Our previously proposed non-truncated sequential algorithm, SeqRDT, was shown to achieve desired error probabilities under a few assumptions on the signal model. In this paper, we show that the proposed truncated sequential algorithm, T-SeqRDT, requires even fewer assumptions on the signal model, while guaranteeing the error probabilities to be below pre-specified levels and at the same time makes a decision faster compared to its optimal fixed-sample-size counterpart, BlockRDT. We derive bounds on the error probabilities and the average stopping times of the algorithm. Via numerical simulations, we compare the performance of T-SeqRDT with SeqRDT, BlockRDT, sequential probability ratio test, and composite sequential probability ratio tests. We also show the robustness of the proposed approach compared with the standard likelihood ratio based approaches. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of quickly detecting an abrupt change in linear regression models. Specifically, an observer sequentially obtains a sequence of observations, whose underlying linear model changes at an unknown time. Moreover, the pre-change linear model is perfectly known by the observer but the post-change linear model is unknown. The observer aims to design an efficient online algorithm to detect the presence of the change via his sequential observations. Based on different assumptions on the change-point, both non-Bayesian and Bayesian problem formulations are considered. In the non-Bayesian setting, the change-point is modeled as a fixed but unknown constant. Two performance metrics, namely the worst case detection delay and the average run length to false alarm, are adopted to evaluate the performance of detection algorithms. In the Bayesian setting, the change-point is modeled as a geometrically distributed random variable. For this case, the average detection delay and the probability of false alarm are used as performance metrics. We propose a novel algorithm, namely the parallel-sum algorithm, for the purpose of change detection. For both setups, we show that the proposed algorithm has a low computational complexity while still offering a good performance in terms of the performance metrics of the respective setting. [ABSTRACT FROM AUTHOR]

We consider a distributed resource allocation problem in networks where each transmitter–receiver pair aims at maximizing its local utility function by adjusting its action matrix, which belongs to a given feasible set. This problem has been addressed recently by applying a matrix exponential learning (MXL) algorithm, which has a very appealing convergence rate. In this learning algorithm, however, each transmitter must know an estimate of the gradient matrix of the local utility. The knowledge of the gradient matrix at the transmitters incurs a high signaling overhead, especially that the matrix size increases with the dimension of the action matrix. In this paper, we therefore, investigate two strategies in order to decrease the informational exchange per iteration of the algorithm. In the first strategy, each receiver sends at each iteration part of the elements of the gradient matrix with respect to a certain probability. In the second strategy, each receiver feeds back “sporadically” the whole gradient matrix. We focus on the analysis of the convergence of the MXL algorithm to optimum under these two strategies. We prove that the algorithm can still converge to optimum almost surely. Upper bounds of the average convergence rate are also derived in both situations with general step-size setting, from which we can clearly see the impact of the incompleteness of the feedback information. The proposed algorithms are applied to solve the energy efficiency maximization problem in a multicarrier multiuser multiple-input and multiple-output (MIMO) network. Simulation results further corroborate our claim. [ABSTRACT FROM AUTHOR]

This paper studies linear distributed estimation of an unknown random parameter vector in a bandwidth-constrained multisensor network. To meet the bandwidth limitations, each sensor converts its observation into a low-dimensional datum via a suitable linear transformation. Then, the fusion center estimates the parameter vector by linearly combining all the received low-dimensional data, aiming at minimizing the estimation mean square error. The main purpose of this paper is to jointly determine the compression dimension of each sensor (referred to as dimension assignment) and design the corresponding compression matrix when the total compression dimensions is limited. Such a joint design problem can be formulated as a rank-constrained optimization problem and it is shown to be NP-hard for the first time. In addition, successive quadratic upper-bound minimization (SQUM), SQUM-block coordinate descent (SQUM-BCD) and nuclear norm regularization (NNR) methods are developed to solve it approximately. Furthermore, we show that any accumulation point of the sequence generated by the SQUM method satisfies the Karush-Kuhn-Tucker conditions of the rank-constrained optimization problem, and the Phase II algorithm of the SQUM-BCD and NNR methods (both are two-phase algorithms and have the same Phase II algorithm) guarantees convergence at least to a stationary point. Numerical experiments illustrate the advantages of the proposed methods compared with the existing method. [ABSTRACT FROM AUTHOR]

Deep structures consisting of many layers of nonlinearities have a high potential of expressing complex relations if properly initialized. Autoencoders play a complementary role in training a deep structure by initializing each layer in a greedy unsupervised manner. Due to the high capacity presented by autoencoders, these structures need to be regularized. While mathematical regularizers (based on weight decay, sparsity, etc.) and structural ones (by way of, e.g., denoising and dropout) have been well studied in the literature, quite a few papers have addressed the problem of training autoencoder with non-smooth regularization. In this paper, we address the problem of training autoencoder with non-smooth regularization. We propose an efficient algorithm and mathematically prove that it is convergent, where the regularizer needs to be proximable. As one of major applications of the proposed method, we get focused on the problem of sparse autoencoders and show that the new training method leads to better disentangling of factors of variation. [ABSTRACT FROM AUTHOR]

Frequency agile radar (FAR) is known to have excellent electronic counter-countermeasures performance and the potential to realize spectrum sharing in dense electromagnetic environments. Many compressed sensing (CS) based algorithms have been developed for joint range and Doppler estimation in FAR. This paper considers theoretical analysis of FAR via CS algorithms. In particular, we analyze the properties of the sensing matrix, which is a highly structured random matrix. We then derive bounds on the number of recoverable targets. Numerical simulations and field experiments validate the theoretical findings and demonstrate the effectiveness of CS approaches to FAR. [ABSTRACT FROM AUTHOR]

This paper considers the problem of minimizing the expected value of a (possibly nonconvex) cost function parameterized by a random (vector) variable, when the expectation cannot be computed accurately (e.g., because the statistics of the random variables are unknown and/or the computational complexity is prohibitive). Classical stochastic gradient methods for solving this problem may suffer from slow convergence. In this paper, we propose a stochastic parallel Successive Convex Approximation-based (best-response) algorithm for general nonconvex stochastic sum-utility optimization problems, which arise naturally in the design of multi-agent networks. The proposed novel decomposition approach enables all users to update their optimization variables in parallel by solving a sequence of strongly convex subproblems, one for each user. Almost sure convergence to stationary points is proved. We then customize the algorithmic framework to solve the stochastic sum rate maximization problem over single-input-single-output (SISO) frequency-selective interference channels, multiple-input-multiple-output (MIMO) interference channels, and MIMO multiple-access channels. Numerical results corroborate that the proposed algorithms can converge faster than state-of-the-art stochastic gradient schemes while achieving the same (or better) sum-rates. [ABSTRACT FROM AUTHOR]

In cognitive radio networks (CRNs), localization of primary users (PUs) and secondary users (SUs) can enable several key capabilities such as location aware routing and power control mechanisms for SUs. Therefore, SUs in a network must accurately locate PUs in order to efficiently use spectrum holes without interfering to the PUs. Accurate localization of PUs in CRN is an important but challenging task due to the unique constraint of CRNs, i.e., the non cooperative nature of PUs making the localization algorithm rely solely on sensing results. In this paper we propose cluster based CRN localization using multidimensional scaling (MDS) that improves accuracy, especially for irregular CRNs. Using the traditional MDS approach leads to low localization accuracy and higher computational complexity. Based on this fact, this paper proposes a novel cluster based multidimensional scaling algorithm for CRN localization (CB-MDS). Furthermore Cramér–Rao lower bound (CRLB) is derived to analyze the performance of the proposed algorithm. Moreover, extensive simulations are performed to confirm that the proposed CB-MDS algorithm is robust to noise and performs better than existing algorithms in attaining the CRLB. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of distributed adaptive estimation of node-specific signals for signal enhancement or noise reduction in wireless sensor networks with multi-sensor nodes. The estimation is performed by a multi-channel Wiener filter (MWF) in which a low-rank approximation based on a generalized eigenvalue decomposition (GEVD) is incorporated. In non-stationary or low-SNR conditions, this GEVD-based MWF has been demonstrated to be more robust than the original MWF. In a centralized realization where a fusion center has access to all the nodes’ sensor signal observations, the network-wide sensor signal correlation matrices and the low-rank approximation can be directly estimated and used to compute the network-wide GEVD-based MWF. However, in this paper, we aim to avoid centralizing the sensor signal observations, in which case the network-wide sensor signal correlation matrices cannot be estimated. To this end, we start from the so-called distributed adaptive node-specific signal estimation (DANSE) algorithm, and include GEVD-based low-rank approximations in the per-node local computations. Remarkably, the new algorithm is able to significantly compress the signal observations transmitted between the nodes, while still converging to the network-wide GEVD-based MWF as if each node would have access to all sensor signal observations, even though the low-rank approximations are applied locally at each node. We provide a theoretical convergence analysis, which shows that the algorithm converges to the network-wide GEVD-based MWF under conditions that are less strict than in the original DANSE algorithm. The convergence and performance of the algorithm are further investigated via numerical simulations. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of the clutter subspace projector estimation in the context of a disturbance composed of a low rank heterogeneous (Compound Gaussian) clutter and white Gaussian noise. In such a context, adaptive processing based on an estimated orthogonal projector onto the clutter subspace (instead of an estimated covariance matrix) requires less samples than classical methods. The clutter subspace estimate is usually derived from the eigenvalue decomposition of a covariance matrix estimate. However, it has been previously shown that a direct maximum likelihood estimator of the clutter subspace projector can be obtained for the considered context. In this paper, we derive two algorithms based on the block majorization-minimization framework to reach this estimator. These algorithms are shown to be computationally faster than the state of the art, with guaranteed convergence. Finally, the performance of the related estimators is illustrated on realistic Space Time Adaptive Processing for airborne radar simulations. [ABSTRACT FROM PUBLISHER]

This paper considers linear downlink transceiver design for the sum power minimization problem with per-user rate constraints in a multi-cell multi-user MIMO system. This non-convex problem is divided into transmit and receive beamforming optimization steps which are iteratively repeated such that the sum power converges. The transmit beamformers are solved using a successive convex approximation method (SCA), whereas the receive beamformers are computed via the linear minimum mean square error (MMSE) criterion. In addition to centralized design, we propose two decentralized algorithms where the transmit and receive beamforming designs are facilitated by a combination of pilot and backhaul signaling. In the first algorithm, sum power is minimized by computing the transmit beamformers at each base station (BS) and the receive beamformers at each user via iterative over-the-air optimization process. BS side processing requires local effective channel state information (CSI) acquired from precoded uplink pilots. In the second algorithm, the transmit and receive beamformers are iteratively optimized at each BS requiring only local CSI obtained from antenna-specific uplink pilots. In both algorithms, transmit beamforming optimization is decoupled among the BSs via primal decomposition method, which requires scalar backhaul information exchange for decentralized processing. The performance of the proposed algorithms is evaluated via numerical examples under different system settings in static and time-correlated channel conditions. [ABSTRACT FROM AUTHOR]

Impulsive processes frequently occur in many fields, such as radar, sonar, communications, audio and speech processing, and biomedical engineering. In this paper, we propose a nonconvex linear regression (NLR) based minimum dispersion beamforming technique for impulsive signals to achieve significant performance improvement over the conventional minimum variance beamformer. The proposed beamformer minimizes the \ellp-norm of the output with p<1 subject to a linear distortionless response constraint, resulting in a difficult nonconvex and nonsmooth optimization problem. The constrained optimization problem is first reduced to a multivariate linear regression via constraint elimination. As a major contribution of this paper, a coordinate descent algorithm (CDA) is devised for solving the resultant NLR problem of \ellp-minimization with p<1 at a computational complexity of \cal O(MN^2), where M is the number of sensors and N is the sample size. At each inner iteration of the CDA, an efficient algorithm is designed to find the global minimum of each subproblem of univariate linear regression. The convergence of the CDA is analyzed. The NLR beamformer with a single constraint is further generalized to the case of multiple linear constraints, which is robust against model mismatch. Simulation results demonstrate the superior performance of nonconvex optimization based beamformer. [ABSTRACT FROM AUTHOR]

This paper develops efficient algorithms for distributed average consensus with quantized communication using the alternating direction method of multipliers (ADMM). We first study the effects of probabilistic and deterministic quantizations on a distributed ADMM algorithm. With probabilistic quantization, this algorithm yields linear convergence to the desired average in the mean sense with a bounded variance. When deterministic quantization is employed, the distributed ADMM converges to a consensus within 3+\lceil\log1+\delta\Omega\rceil iterations where \delta>0 depends on the network topology and \Omega is a polynomial fraction depending on the quantization resolution, the agents’ data, and the network topology. A tight upper bound on the consensus error is also obtained, which depends only on the quantization resolution and the average degree of the graph. This bound is much preferred in large scale networks over existing algorithms whose consensus errors are increasing in the range of agents’ data, the quantization resolution, and the number of agents. We finally propose our algorithm which combines both probabilistic and deterministic quantizations. Simulations show that the consensus error of our algorithm is typically less than one quantization resolution for all connected networks where agents’ data can be of arbitrary magnitudes. [ABSTRACT FROM AUTHOR]

For self-localization, a detailed and reliable map of the environment can be used to relate sensor data to static features with known locations. This paper presents a method for construction of detailed radar maps that describe the expected intensity of detections. Specifically, the measurements are modelled by an inhomogeneous Poisson process with a spatial intensity function given by the sum of a constant clutter level and an unnormalized Gaussian mixture. A substantial difficulty with radar mapping is the presence of data association uncertainties, i.e., the unknown associations between measurements and landmarks. In this paper, the association variables are introduced as hidden variables in a variational Bayesian expectation maximization (VBEM) framework, resulting in a computationally efficient mapping algorithm that enables a joint estimation of the number of landmarks and their parameters. [ABSTRACT FROM PUBLISHER]