Search

We propose a novel adaptive kernel-based regression method for complex-valued signals: the generalized complex-valued kernel least-mean-square (gCKLMS). We borrow from the new results on widely linear reproducing kernel Hilbert space (WL-RKHS) for nonlinear regression and complex-valued signals, recently proposed by the authors. This paper shows that in the adaptive version of the kernel regression for complex-valued signals we need to include another kernel term, the so-called pseudo-kernel. This new solution is endowed with better representation capabilities in complex-valued fields since it can efficiently decouple the learning of the real and the imaginary part. Also, we review previous realizations of the complex KLMS algorithm and its augmented version to prove that they can be rewritten as particular cases of the gCKLMS. Furthermore, important conclusions on the design of the kernels are drawn that help to greatly improve the convergence of the algorithms. In the experiments, we revisit the nonlinear channel equalization problem to highlight the better convergence of the gCKLMS compared to previous solutions. Also, the flexibility of the proposed generalized approach is tested in a second experiment with non-independent real and imaginary parts. The results illustrate the significant performance improvements of the gCKLMS approach when the complex-valued signals have different properties for the real and imaginary parts. [ABSTRACT FROM AUTHOR]

Sparse coding refers to the pursuit of the sparsest representation of a signal in a typically overcomplete dictionary. From a Bayesian perspective, sparse coding provides a maximum a posteriori estimate of the unknown vector under a sparse prior. In this paper, we suggest enhancing the performance of sparse coding algorithms by a deliberate and controlled contamination of the input with random noise, a phenomenon known as stochastic resonance. The proposed method adds controlled noise to the input and estimates a sparse representation from the perturbed signal. A set of such solutions is then obtained by projecting the original input signal onto the recovered set of supports. We present two variants of the described method, which differ in their final step. The first is a provably convergent approximation to the minimum mean square error (MMSE) estimator, relying on the generative model and applying a weighted average over the recovered solutions. The second is a relaxed variant of the former that simply applies an empirical mean. We show that both methods provide a computationally efficient approximation to the MMSE estimator, which is typically intractable to compute. We demonstrate our findings empirically and provide a theoretical analysis of our method under several different cases. [ABSTRACT FROM AUTHOR]

Sampling from the lattice Gaussian distribution has emerged as an important problem in coding, decoding, and cryptography. In this paper, lattice reduction technique is adopted to Gibbs sampler for lattice Gaussian sampling. First, with respect to lattice Gaussian distribution, we show the convergence rate of systematic scan Gibbs sampling is characterized by the Hirschfeld-Gebelein-Rényi maximal correlation among the multivariate of being sampled. Then, Lattice-reduction-aided Gibbs algorithm is proposed to sample from an equivalent lattice Gaussian distribution but with less correlated multivariate, thus leading to a better Markov mixing. After that, we extend the proposed lattice-reduction-aided Gibbs sampling to lattice decoding, where the choice of the standard deviation for the sampling is fully investigated. A customized solution that suits for each specific decoding case by Euclidean distance is given, which results in a better tradeoff between Markov mixing and sampler decoding. Based on it, a startup mechanism is also proposed for Gibbs sampler decoding, where decoding complexity can be reduced without performance loss. Moreover, the recycling Gibbs sampling that exploits the potential of samples is also considered to improve the decoding performance in lattice decoding. Simulation results based on large-scale uncoded multiple-input multiple-output detection are presented to confirm the performance gain and complexity reduction. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel sparse signal recovery algorithm called the trainable iterative soft thresholding algorithm (TISTA). The proposed algorithm consists of two estimation units: a linear estimation unit and a minimum mean squared error (MMSE) estimator based shrinkage unit. The error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The remarkable feature of the proposed scheme is that TISTA includes adjustable variables that control step size and the error variance for the MMSE shrinkage function. The variables are adjusted by standard deep learning techniques. The number of trainable variables of TISTA is nearly equal to the number of iteration rounds and is much smaller than that of known learnable sparse signal recovery algorithms. This feature leads to highly stable and fast training processes of TISTA. Computer experiments show that TISTA is applicable to various classes of sensing matrices, such as Gaussian matrices, binary matrices, and matrices with large condition numbers. Numerical results also demonstrate that, in many cases, TISTA provides significantly faster convergence than approximate message passing (AMP) and the learned iterative shrinkage thresholding algorithm and also outperforms orthogonal AMP in the NMSE performance. [ABSTRACT FROM AUTHOR]

This paper proposes a decentralized algorithm for solving a consensus optimization problem defined in a static networked multi-agent system, where the local objective functions have the smooth+nonsmooth composite form. Examples of such problems include decentralized constrained quadratic programming and compressed sensing problems, as well as many regularization problems arising in inverse problems, signal processing, and machine learning, which have decentralized applications. This paper addresses the need for efficient decentralized algorithms that take advantages of proximal operations for the nonsmooth terms. We propose a proximal gradient exact first-order algorithm (PG-EXTRA) that utilizes the composite structure and has the best known convergence rate. It is a nontrivial extension to the recent algorithm EXTRA. At each iteration, each agent locally computes a gradient of the smooth part of its objective and a proximal map of the nonsmooth part, as well as exchanges information with its neighbors. The algorithm is “exact” in the sense that an exact consensus minimizer can be obtained with a fixed step size, whereas most previous methods must use diminishing step sizes. When the smooth part has Lipschitz gradients, PG-EXTRA has an ergodic convergence rate of O\left(1\over k\right) in terms of the first-order optimality residual. When the smooth part vanishes, PG-EXTRA reduces to P-EXTRA, an algorithm without the gradients (so no “G” in the name), which has a slightly improved convergence rate at o\left(1\over k\right) in a standard (non-ergodic) sense. Numerical experiments demonstrate effectiveness of PG-EXTRA and validate our convergence results [ABSTRACT FROM PUBLISHER]

There are two fundamentally different fronthaul techniques in the downlink communication of cloud radio access network (C-RAN): the data-sharing strategy and the compression-based strategy. Under the former strategy, each user's message is multicast from the central processor (CP) to all the serving remote radio heads (RRHs) over the fronthaul network, which then cooperatively serve the users through joint beamforming; while under the latter strategy, the user messages are first beamformed then quantized at the CP, and the compressed signal is unicast to the corresponding RRH, which then decompresses its received signal for wireless transmission. Previous works show that in general the compression-based strategy outperforms the data-sharing strategy. This paper, on the other hand, points out that in a C-RAN model where the RRHs are connected to the CP via multihop routers, data-sharing can be superior to compression if the network coding technique is adopted for multicasting user messages to the cooperating RRHs, and the RRH's beamforming vectors, the user-RRH association, and the network coding design over the fronthaul network are jointly optimized based on the techniques of sparse optimization and successive convex approximation. This is in comparison to the compression-based strategy, where information is unicast over the fronthaul network by simple routing, and the RRH's compression noise covariance and beamforming vectors, as well as the routing strategy over the fronthaul network are jointly optimized based on the successive convex approximation technique. The observed gain in overall network throughput is due to that information multicast is more efficient than information unicast over the multihop fronthaul of a C-RAN. [ABSTRACT FROM PUBLISHER]

Designing and implementing systems as an interconnection of smaller subsystems is a common practice for modularity and standardization of components and design algorithms. Although not typically cast in this framework, many of these approaches can be viewed within the mathematical context of functional composition. This paper re-interprets and generalizes within the functional composition framework one such approach known as filter sharpening, i.e., interconnecting filter modules which have significant approximation error in order to obtain improved filter characteristics. More specifically, filter sharpening is approached by determining the composing polynomial to minimize the infinity-norm of the approximation error, utilizing the First Algorithm of Remez. This is applied both to sharpening for FIR, even-symmetric filters and for the more general case of subfilters that have complex-valued frequency responses including causal IIR filters and for continuous-time filters. Within the framework of functional composition, this paper also explores the use of functional decomposition to approximate a desired system as a composition of simpler functions based on a two-norm on the approximation error. Among the potential advantages of this decomposition is the ability for modular implementation in which the inner component of the functional decomposition represents the subfilters and the outer the interconnection. [ABSTRACT FROM AUTHOR]

Classification with a sparsity constraint on the solution plays a central role in many high dimensional signal processing applications. In some cases, the features can be grouped together, so that entire subsets of features can be selected or discarded. In many applications, however, this can be too restrictive. In this paper, we are interested in a less restrictive form of structured sparse feature selection: We assume that while features can be grouped according to some notion of similarity, not all features in a group need be selected for the task at hand. The Sparse Group Lasso (SGL) was proposed to solve problems of this form. The main contributions of this paper are a new procedure called Sparse Overlapping Group (SOG) lasso, an extension to the SGL to overlapping groups and theoretical sample complexity bounds for the same. We establish model selection error bounds that specializes to many other cases. We experimentally validate our proposed method on both real and toy datasets. [ABSTRACT FROM AUTHOR]

Greed is good. However, the tighter you squeeze, the less you have. In this paper, a less greedy algorithm for sparse signal reconstruction in compressive sensing, named orthogonal matching pursuit with thresholding is studied. Using the global 2-coherence, which provides a “bridge” between the well known mutual coherence and the restricted isometry constant, the performance of orthogonal matching pursuit with thresholding is analyzed and more general results for sparse signal reconstruction are obtained. It is also shown that given the same assumption on the coherence index and the restricted isometry constant as required for orthogonal matching pursuit, the thresholding variation gives exactly the same reconstruction performance with significantly less complexity. [ABSTRACT FROM PUBLISHER]

This paper studies the 2D localization problem of a sensor network given anchor node positions in a common global coordinate frame and relative position measurements in local coordinate frames between node pairs. It is assumed that the local coordinate frames of different sensors have different orientations and the orientation difference with respect to the global coordinate frame are not known. In terms of graph connectivity, a necessary and sufficient condition is obtained for self-localizability that leads to a fully distributed localization algorithm. Moreover, a distributed verification algorithm is developed to check the graph connectivity condition, which can terminate successfully when the sensor network is self-localizable. Finally, a fully distributed, linear, and iterative algorithm based on the complex-valued Laplacian associated with the sensor network is proposed, which converges globally and gives the correct localization result. [ABSTRACT FROM PUBLISHER]

Bayesian filtering aims at estimating sequentially a hidden process from an observed one. In particular, sequential Monte Carlo (SMC) techniques propagate in time weighted trajectories which represent the posterior probability density function (pdf) of the hidden process given the available observations. On the other hand, conditional Monte Carlo (CMC) is a variance reduction technique which replaces the estimator of a moment of interest by its conditional expectation given another variable. In this paper, we show that up to some adaptations, one can make use of the time recursive nature of SMC algorithms in order to propose natural temporal CMC estimators of some point estimates of the hidden process, which outperform the associated crude Monte Carlo (MC) estimator whatever the number of samples. We next show that our Bayesian CMC estimators can be computed exactly, or approximated efficiently, in some hidden Markov chain (HMC) models; in some jump Markov state-space systems (JMSS); as well as in multitarget filtering. Finally our algorithms are validated via simulations. [ABSTRACT FROM PUBLISHER]

We design an efficient robust multi-group multicast beamforming scheme for massive multiple-input multiple-output (MIMO) systems. Assuming only estimates of the channel covariance matrices are available at the base station with a bounded error, we formulate the robust quality-of-service (QoS) problem, which is to minimize the transmit power subject to the worst-case minimum signal-to-interference-plus-noise-ratio (SINR) guarantee. We directly solve the worst-case SINR problem and convert the robust QoS constraint into a number of non-convex constraints. Based on the recent convergence result of the alternating direction method of multipliers (ADMM) for non-convex problems, we develop an ADMM-based fast algorithm to directly tackle the reformulated non-convex problem with a convergence guarantee. The algorithm contains two layers of ADMM procedures. We design the outer-layer ADMM to decompose the problem into three convex subproblems and solve them alternatingly. We further develop an inner-layer consensus-ADMM-based algorithm to efficiently solve one subproblem. By exploring each subproblem structure and developing the special optimization techniques, we obtain closed-form or semi-closed-form solutions to each subproblem. These results lead to a fast iterative algorithm, which is guaranteed to converge to a stationary point of the original robust QoS problem. Simulation shows that our proposed algorithm provides a favorable performance compared with existing alternative methods with magnitudes of computational complexity reduction. [ABSTRACT FROM AUTHOR]

This paper considers the mean-reverting portfolio (MRP) design problem arising from statistical arbitrage (a.k.a. pairs trading) in the financial markets. It aims at designing a portfolio of underlying assets by optimizing the mean reversion strength of the portfolio, while taking into consideration the portfolio variance and an investment budget constraint. Several specific design problems are considered based on different mean reversion criteria. Efficient algorithms are proposed to solve the problems. Numerical results on both synthetic and market data show that the proposed MRP design methods can generate consistent profits and outperform the traditional design methods and the benchmark methods in the literature. [ABSTRACT FROM AUTHOR]

This paper develops a novel algorithm, termed SPARse Truncated Amplitude flow (SPARTA), to reconstruct a sparse signal from a small number of magnitude-only measurements. It deals with what is also known as sparse phase retrieval (PR), which is NP-hard in general and emerges in many science and engineering applications. Upon formulating sparse PR as an amplitude-based nonconvex optimization task, SPARTA works iteratively in two stages: In stage one, the support of the underlying sparse signal is recovered using an analytically well-justified rule, and subsequently a sparse orthogonality-promoting initialization is obtained via power iterations restricted on the support; and in the second stage, the initialization is successively refined by means of hard thresholding based gradient-type iterations. SPARTA is a simple yet effective, scalable, and fast sparse PR solver. On the theoretical side, for any n-dimensional k-sparse (k ≪ n) signal x with minimum (in modulus) nonzero entries on the order of (1/√k)||x||2, SPARTA recovers the signal exactly (up to a global unimodular constant) from about k2 log n random Gaussian measurements with high probability. Furthermore, SPARTA incurs computational complexity on the order of k2 n log n with total runtime proportional to the time required to read the data, which improves upon the state of the art by at least a factor of k. Finally, SPARTA is robust against additive noise of bounded support. Extensive numerical tests corroborate markedly improved recovery performance and speedups of SPARTA relative to existing alternatives. [ABSTRACT FROM AUTHOR]

In this paper, we present an algorithm for the sparse signal recovery problem that incorporates damped Gaussian generalized approximate message passing (GGAMP) into expectation-maximization-based sparse Bayesian learning (SBL). In particular, GGAMP is used to implement the E-step in SBL in place of matrix inversion, leveraging the fact that GGAMP is guaranteed to converge with appropriate damping. The resulting GGAMP-SBL algorithm is much more robust to arbitrary measurement matrix A than the standard damped GAMP algorithm while being much lower complexity than the standard SBL algorithm. We then extend the approach from the single measurement vector case to the temporally correlated multiple measurement vector case, leading to the GGAMP-TSBL algorithm. We verify the robustness and computational advantages of the proposed algorithms through numerical experiments. [ABSTRACT FROM AUTHOR]

This paper considers coordinated multicast beamforming in a multicell multigroup multiple-input single-output system. Each base station (BS) serves multiple groups of users by forming a single beam with common information per group. We propose centralized and distributed beamforming algorithms for two different optimization targets. The first objective is to minimize the total transmission power of all the BSs while guaranteeing the user-specific minimum quality-of-service targets. The semidefinite relaxation (SDR) method is used to approximate the nonconvex multicast problem as a semidefinite program (SDP), which is solvable via centralized processing. Subsequently, two alternative distributed methods are proposed. The first approach turns the SDP into a two-level optimization via primal decomposition. At the higher level, intercell interference powers are optimized for fixed beamformers, whereas the lower level locally optimizes the beamformers by minimizing BS-specific transmit powers for the given intercell interference constraints. The second distributed solution is enabled via an alternating direction method of multipliers, where the intercell interference optimization is divided into a local and a global optimization by forcing the equality via consistency constraints. We further propose a centralized and a simple distributed beamforming design for the signal-to-interference-plus-noise ratio (SINR) balancing problem in which the minimum SINR among the users is maximized with given per-BS power constraints. This problem is solved via the bisection method as a series of SDP feasibility problems. The simulation results show the superiority of the proposed coordinated beamforming algorithms over traditional noncoordinated transmission schemes, and illustrate the fast convergence of the distributed methods. [ABSTRACT FROM PUBLISHER]

In this paper, we study the target tracking problem in a wireless sensor network. A sensor receives a measurement from an energy emitting target and employs binary quantization to the received measurement to generate its decision. A sinusoidal waveform with a certain duration is then used to transmit the sensor decision to the fusion center (FC). All sensor decisions are transmitted to the FC over erroneous wireless channels based on a time division multiple access scheme. We introduce the proportional time allocation (PTA) algorithm where at each time step of tracking, PTA jointly determines the sensors binary quantization thresholds and their time allocations devoted for the transmissions of binary sensor decisions. Simulation results show that, PTA optimally and dynamically distributes the available transmission time among the sensors near the target so that the decisions of such sensors become less subject to channel errors, and turns off the non-informative sensors located far away from the target. Hence, PTA both saves from the number of sensors transmitting to the FC and provides better estimation performance as compared to ad hoc equal time allocation approaches. [ABSTRACT FROM PUBLISHER]

This paper considers unimodular sequence synthesis under similarity constraint for both the continuous and discrete phase cases. A computationally efficient iterative algorithm for the continuous phase case (IA-CPC) is proposed to sequentially optimize the quadratic objective function. The quadratic optimization problem is turned into multiple one-dimensional optimization problems with closed-form solutions. For the discrete phase case, we present an iterative block optimization algorithm. Specifically, we partition the design variables into $K$ blocks, and then, we sequentially optimize each block via exhaustive search while fixing the remaining $K-1$ blocks. Finally, we evaluate the computational costs and performance gains of the proposed algorithms in comparison with power method-like and semidefinite relaxation related techniques. [ABSTRACT FROM PUBLISHER]

The paper is concerned with stabilizing the family of adaptive filtering algorithms based on minimizing the $2Lth$ moment of the estimation error, with $L$ being an integer greater than 1. Stabilization is attained via a proposed normalization of the algorithm. Mean square stability of the normalized algorithm is proved for a Markov plant for all $L > 1$ . Transient and steady-state performances of the algorithm are analyzed for a time-invariant plant. Expressions are derived for the steady-state misadjustment and convergence time. Tradeoff between the transient and steady-state performances is evaluated. Dependence of this tradeoff on the value of L is studied. This tradeoff is compared with the tradeoff of the NLMS algorithm. The proposed algorithm is dramatically superior to the NLMS algorithm for sub-Gaussian noise. The superiority increases with L, initial mean-square deviation and signal-to-noise ratio. Analytical results are supported by simulations. [ABSTRACT FROM PUBLISHER]

Consider the single-group multicast beamforming problem, where multiple users receive the same data stream simultaneously from a single transmitter. The problem is NP-hard and all existing algorithms for the problem either find suboptimal approximate or local stationary solutions. In this paper, we propose an efficient branch-and-bound algorithm for the problem that is guaranteed to find its global solution. To the best of our knowledge, our proposed algorithm is the first tailored global algorithm for the single-group multicast beamforming problem. Simulation results show that our proposed algorithm is computationally efficient (albeit its theoretical worst-case iteration complexity is exponential with respect to the number of receivers) and it significantly outperforms a state-of-the-art general-purpose global optimization solver called Baron. Our proposed algorithm provides an important benchmark for performance evaluation of existing algorithms for the same problem. By using it as the benchmark, we show that two state-of-the-art algorithms, semidefinite relaxation algorithm and successive linear approximation algorithm, work well when the problem dimension (i.e., the number of antennas at the transmitter and the number of receivers) is small but their performance deteriorates quickly as the problem dimension increases. [ABSTRACT FROM PUBLISHER]

The low-rank matrix completion problem is fundamental to a number of tasks in data mining, machine learning, and signal processing. This paper considers the problem of adaptive matrix completion in time-varying scenarios. Given a sequence of incomplete and noise-corrupted matrices, the goal is to recover and track the underlying low rank matrices. Motivated from the classical least-mean square (LMS) algorithms for adaptive filtering, three LMS-like algorithms are proposed for estimating and tracking low-rank matrices. Performance of the proposed algorithms is provided in form of nonasymptotic bounds on the tracking mean-square error. Tracking performance of the algorithms is also studied via detailed simulations over real-world datasets. [ABSTRACT FROM PUBLISHER]

Based on the minimum noise subspace (MNS) method previously introduced in the context of blind channel identification, generalized minimum noise subspace (GMNS) is proposed in this paper for array processing that generalizes MNS with respect to the availability of only a fixed number of parallel computing units. Different batch and adaptive algorithms are then introduced for fast and parallel computation of signal (principal) and noise (minor) subspaces. The computational complexity of GMNS and its related estimation accuracy are investigated by simulated experiments and a real-life experiment in radio astronomy. It is shown that GMNS represents an excellent tradeoff between the computational gain and the subspace estimation accuracy, as compared to several standard subspace methods. [ABSTRACT FROM PUBLISHER]

Neural networks with rectified linear unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing results for learning ReLU networks either pose assumptions on the underlying data distribution being, e.g., Gaussian, or require the network size and/or training size to be sufficiently large. In this context, the problem of learning a two-layer ReLU network is approached in a binary classification setting, where the data are linearly separable and a hinge loss criterion is adopted. Leveraging the power of random noise perturbation, this paper presents a novel stochastic gradient descent (SGD) algorithm, which can provably train any single-hidden-layer ReLU network to attain global optimality, despite the presence of infinitely many bad local minima, maxima, and saddle points in general. This result is the first of its kind, requiring no assumptions on the data distribution, training/network size, or initialization. Convergence of the resultant iterative algorithm to a global minimum is analyzed by establishing both an upper bound and a lower bound on the number of non-zero updates to be performed. Moreover, generalization guarantees are developed for ReLU networks trained with the novel SGD leveraging classic compression bounds. These guarantees highlight a key difference (at least in the worst case) between reliably learning a ReLU network as well as a leaky ReLU network in terms of sample complexity. Numerical tests using both synthetic data and real images validate the effectiveness of the algorithm and the practical merits of the theory. [ABSTRACT FROM AUTHOR]

We deal with the classical problem of testing two simple statistical hypotheses but, as a new element, it is assumed that the data vector is observed after an unknown permutation of its entries. What is the fundamental limit for the detection performance in this case? How much information for detection is contained in the entry values and how much in their positions? In the first part of this paper, we answer these questions. In the second part, we focus on practical algorithms. A low-complexity detector solves the detection problem without attempting to estimate the permutation. A modified version of the auction algorithm is then considered, and two greedy algorithms with affordable worst case complexity are presented. The detection operational characteristics of these detectors are investigated by computer experiments. The problem we address is referred to as unlabeled detection and is motivated by large sensor network applications, but applications are also foreseen in different fields, including image processing, social sensing, genome research, and molecular communication. [ABSTRACT FROM AUTHOR]

Massive multiple-input-multiple-output (MIMO) scheme promises high spectral efficiency through the employment of large scale antenna arrays in base stations. In time division duplexed implementations, co-channel mobile terminals transmit training information such that base stations can estimate and exploit channel state information to spatially multiplex these users. In the conventional approach, the optimal choice for training length was shown to be equal to the number of users, $K$. In this paper, we propose a new semiblind framework, named as “MIMO Compressed Training,” which utilizes information symbols in addition to training symbols for adaptive spatial multiplexing. We show that this framework enables us to reduce (compress) the training length down to a value close to $\log _2(K)$ , i.e., the logarithm of the number of users, without any sparsity assumptions on the channel matrix. We also derive a prescription for the required packet length for proper training. The framework is built upon some convex optimization settings that enable efficient and reliable algorithm implementations. The numerical experiments demonstrate the strong potential of the proposed approach in terms of increasing the number of users per cell and improving the link quality. [ABSTRACT FROM AUTHOR]

In CANDECOMP/PARAFAC tensor decomposition, degeneracy often occurs in some difficult scenarios, especially, when the rank exceeds the tensor dimension, or when the loading components are highly collinear in several or all modes, or when CPD does not have an optimal solution. In such cases, norms of some rank-1 tensors become significantly large and cancel each other. This makes algorithms getting stuck in local minima while running a huge number of iterations does not improve the decomposition. In this paper, we propose an error preservation correction method to deal with such problem. Our aim is to seek an alternative tensor, which preserves the approximation error, but norms of rank-1 tensor components of the new tensor are minimized. Alternating and all-at-once correction algorithms have been developed for the problem. In addition, we propose a novel CPD with a bound constraint on the norm of the rank-one tensors. The method can be useful for decomposing tensors that cannot be performed by traditional algorithms. Finally, we demonstrate an application of the proposed method in image denoising and decomposition of the weight tensors in convolutional neural networks. [ABSTRACT FROM AUTHOR]

This paper proposes a new method for impulsive source separation (deinterleaving) in multi-input single-output systems based purely on impulse spacing. The impulse spacing for each source is modeled as a renewal process. To classify sources, an algorithm is developed to maximize the likelihood of impulse assignment based on the impulse spacing distributions. A theoretical lower bound on performance is derived and the algorithm is shown to get reasonably close to the bound. The algorithm is extended for use when the impulse spacing distribution parameters are unknown. Performance is evaluated on multiple application examples. [ABSTRACT FROM AUTHOR]

Next generation wireless networks will exploit the large amount of spectrum available at millimeter wave (mmWave) frequencies. The design of mmWave systems, however, is challenging due to the strict power, cost, and hardware constraints at higher bandwidths. To achieve a good SNR for communication, mmWave systems use large antenna arrays. Beamforming with highly directional beams is one way to use the antennas. As the channel changes over time, the beams that maximize the SNR have to be estimated quickly to reduce the training overhead. Prior research has exploited the observation that mmWave channels are sparse to perform compressed sensing (CS) based beam alignment with few channel measurements. Most of the existing CS-based algorithms, however, assume perfect synchronization and fail in the presence of carrier frequency offset (CFO). This paper presents Swift-Link, a fast beam alignment algorithm that is robust against the offset. Swift-Link includes a novel randomized beam training sequence that minimizes the beam alignment errors due to CFO and a low-complexity algorithm that corrects these errors. Even with strict hardware constraints, our algorithm uses fewer channel measurements than comparable CS algorithms and has analytical guarantees. Swift-Link requires a small output dynamic range at the analog-to-digital converter compared to beam-scanning techniques. [ABSTRACT FROM AUTHOR]

In this paper, a computationally efficient algorithm for Bayesian joint change point detection (CPD) in multiple time series is presented. The data generation model includes a number of change configurations (CC), each affecting a unique subset of the time series, which introduces correlation between the positions of change points (CPs) in the monitored time series. The inference objective is to identify joint changes and the associated CC. The algorithm consists of two stages: First, a univariate CPD algorithm is applied separately to each of the involved time series. The outcomes of this step are maximum a posteriori (MAP) detected CPs and posterior distributions of CPs conditioned on the MAP CPs. These outcomes are used in combination to approximate the posterior for the CCs. In the second algorithm stage, dynamic programming is used to find the maxima of this approximate CC posterior. The algorithm is applied to synthetic data, and it is shown to be both significantly faster and more accurate compared to a previously proposed algorithm designed to solve similar problems. Also, the initial algorithm is extended with steps from the maximization–maximization algorithm, which allows the hyperparameters of the data generation model to be estimated jointly with the CCs, and we show that these estimates coincide with estimates obtained from a Markov chain Monte Carlo algorithm. [ABSTRACT FROM AUTHOR]

Multifractal analysis has become a powerful signal processing tool that characterizes signals or images via the fluctuations of their pointwise regularity, quantified theoretically by the so-called multifractal spectrum. The practical estimation of the multifractal spectrum fundamentally relies on exploiting the scale dependence of statistical properties of appropriate multiscale quantities, such as wavelet leaders, that can be robustly computed from discrete data. Despite successes of multifractal analysis in various real-world applications, current estimation procedures remain essentially limited to providing concave upper-bound estimates, while there is a priori no reason for the multifractal spectrum to be a concave function. This paper addresses this severe practical limitation and proposes a novel formalism for multifractal analysis that enables nonconcave multifractal spectra to be estimated in a stable way. The key contributions reside in the development and theoretical study of a generalized multifractal formalism to assess the multiscale statistics of wavelet leaders, and in devising a practical algorithm that permits this formalism to be applied to real-world data, allowing for the estimation of nonconcave multifractal spectra. Numerical experiments are conducted on several synthetic multifractal processes as well as on a real-world remote-sensing image and demonstrate the benefits of the proposed multifractal formalism over the state of the art. [ABSTRACT FROM AUTHOR]

In this paper, four iterative algorithms for learning analysis operators are presented. They are built upon the same optimization principle underlying both Analysis K-SVD and Analysis SimCO. The forward and sequential analysis operator learning (AOL) algorithms are based on projected gradient descent with optimally chosen step size. The implicit AOL algorithm is inspired by the implicit Euler scheme for solving ordinary differential equations and does not require to choose a step size. The fourth algorithm, singular value AOL, uses a similar strategy as Analysis K-SVD while avoiding its high computational cost. All algorithms are proven to decrease or preserve the target function in each step and a characterization of their stationary points is provided. Further they are tested on synthetic and image data, compared to Analysis SimCO and found to give better recovery rates and faster decay of the objective function, respectively. In a final denoising experiment the presented algorithms are again shown to perform similar to or better than the state-of-the-art algorithm ASimCO. [ABSTRACT FROM AUTHOR]

This paper considers the reconstruction of a discrete-valued random vector from possibly underdetermined linear measurements using sum-of-absolute-value (SOAV) optimization. The proposed algorithm, referred to as discreteness-aware approximate message passing (DAMP), is based on the idea of approximate message passing (AMP), which has been originally proposed for compressed sensing. The DAMP algorithm has low computational complexity and its performance in the large system limit can be predicted analytically via state evolution framework, where we provide a condition for the exact reconstruction with DAMP in the noise-free case. From the analysis, we also propose a method to determine the parameters of the SOAV optimization. Moreover, based on the state evolution, we provide Bayes optimal DAMP, which has the minimum mean-square-error at each iteration of the algorithm. Simulation results show that the DAMP algorithms can reconstruct the discrete-valued vector from underdetermined linear measurements and the empirical performance agrees with our theoretical results in large-scale systems. When the problem size is not large enough, the SOAV optimization with the proposed parameters can achieve better performance than the DAMP algorithms for high signal-to-noise ratio. [ABSTRACT FROM AUTHOR]

The 3GPP suggests to combine dual polarized (DP) antenna arrays with the double directional (DD) channel model for downlink channel estimation. This combination strikes a good balance between high-capacity communications and parsimonious channel modeling, and also brings limited feedback schemes for downlink channel state information within reach—since such channel can be fully characterized by several key parameters. However, most existing channel estimation work under the DD model has not yet considered DP arrays, perhaps because of the complex array manifold and the resulting difficulty in algorithm design. In this paper, we first reveal that the DD channel with DP arrays at the transmitter and receiver can be naturally modeled as a low-rank tensor, and thus the key parameters of the channel can be effectively estimated via tensor decomposition algorithms. On the theory side, we show that the DD–DP parameters are identifiable under mild conditions, by leveraging identifiability of low-rank tensors. Furthermore, a compressed tensor decomposition algorithm is developed for alleviating the downlink training overhead. We show that, by using judiciously designed pilot structure, the channel parameters are still guaranteed to be identified via the compressed tensor decomposition formulation even when the size of the pilot sequence is much smaller than what is needed for conventional channel identification methods, such as linear least squares and matched filtering. Extensive simulations are employed to showcase the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

The problem of reconstructing a sparse signal vector from magnitude-only measurements (a.k.a. compressive phase retrieval) emerges in diverse applications, but it is NP-hard in general. Building on recent advances in nonconvex optimization, this paper puts forth a new algorithm that is termed compressive reweighted amplitude flow (CRAF) and abbreviated as CRAF, for compressive phase retrieval. Specifically, CRAF operates in two stages. The first stage seeks a sparse initial guess via a new spectral procedure. In the second stage, CRAF implements a few hard thresholding based iterations using reweighted gradients. When there are sufficient measurements, CRAF provably recovers the underlying signal vector exactly with high probability under suitable conditions. Moreover, its sample complexity coincides with that of state-of-the-art procedures. Finally, numerical tests showcase improved performance of the new spectral initialization, as well as improved exact recovery relative to competing alternatives. Empirically, the number of measurements required for exact recovery by CRAF is about 40% less than those of competing alternatives in our experiments using real images. [ABSTRACT FROM AUTHOR]

In pulsed Doppler radars, subsampling causes incorrect estimates of the radial velocity of the detected targets whenever its related Doppler frequency is greater than the pulse repetition frequency of the radar. The classic Chinese remainder theorem is a well-known method to overcome this problem. On the other hand, the robust Chinese remainder theorem is a search-based algorithm that, under some conditions, can guarantee the correct estimation of the Doppler frequency. In this paper, we have derived the exact probability of detection for both methods applied in a pulsed Doppler radar. Simulations have been compared to the analytical results and have shown an excellent agreement. [ABSTRACT FROM AUTHOR]

Phase retrieval is an inverse problem which consists in recovering an unknown signal from a set of absolute squared projections. Recently, gradient descent algorithms have been developed to solve this problem. However, their optimization cost functions are non-convex and non-smooth. To address the non-smoothness of the cost function, some of these methods use truncation thresholds to calculate a truncated step gradient direction. But, the truncation requires designing parameters to obtain a desired performance in the phase recovery, which drastically modifies the search direction update, increasing the sampling complexity. Therefore, this paper develops the Phase Retrieval Smoothing Conjugate Gradient method (PR-SCG) which uses a smoothing function to retrieve the signal. PR-SCG is based on the smooth-ing projected gradient method which is useful for non-convex optimization problems. PR-SCG uses a nonlinear conjugate gradient of the smoothing function as the search direction to accelerate the convergence. Furthermore, the incremental Stochastic Smoothing Phase Retrieval algorithm (SSPR) is developed. SSPR involves a single equation per iteration which results in a simple, scalable, and fast approach useful when the size of the signal is large. Also, it is shown that SSPR converges linearly to the true signal, up to a global unimodular constant. Additionally, the proposed methods do not require truncation parameters. Simulation results are provided to validate the efficiency of PR-SCG and SSPR compared to existing phase retrieval algorithms. It is shown that PR-SCG and SSPR are able to reduce the number of measurements and iterations to recover the phase, compared with recently developed algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we propose an online learning algorithm for optimal execution in the limit order book of a financial asset. Given a certain number of shares to sell and an allocated time window to complete the transaction, the proposed algorithm dynamically learns the optimal number of shares to sell via market orders at prespecified time slots within the allocated time interval. We model this problem as a Markov Decision Process (MDP), which is then solved by dynamic programming. First, we prove that the optimal policy has a specific form, which requires either selling no shares or the maximum allowed amount of shares at each time slot. Then, we consider the learning problem, in which the state transition probabilities are unknown and need to be learned on the fly. We propose a learning algorithm that exploits the form of the optimal policy when choosing the amount to trade. Interestingly, this algorithm achieves bounded regret with respect to the optimal policy computed based on the complete knowledge of the market dynamics. Our numerical results on several finance datasets show that the proposed algorithm performs significantly better than the traditional Q-learning algorithm by exploiting the structure of the problem. [ABSTRACT FROM AUTHOR]

The linear canonical transform (LCT) has been shown to be one of the most powerful tools in signal processing, and in this paper, we propose an adaptive approach for the computation of the discrete LCT (DLCT), termed the sliding discrete linear canonical transform (SDLCT). First, we introduce a scheme for the single-point DLCT, which can effectively calculate a single or a few linear canonical spectra. Second, the SDLCT is proposed based on an iterative algorithm to meet the requirements of online spectral analysis when only a subset of $N$ frequencies are required from an $\tilde{N}\hbox{--}$ point discrete LCT ($N\leq \tilde{N}$). The additivity and reversibility properties of the proposed algorithms are also discussed in detail. Third, the DLCT convolution operation is obtained to reduce the spectral leakage of the proposed algorithm, and time-domain windowing is implemented via frequency-domain convolution. Finally, we present two methods to assess performance with regard to computational complexity and precision and to show the correctness of the derived results. [ABSTRACT FROM AUTHOR]

A set-membership affine projection algorithm is proposed that can estimate a complex-valued channel matrix using a set of complex-valued pilots in the presence of additive white Gaussian noise. It is shown that the algorithm converges faster than the well-known set-membership normalized least mean square algorithm (SM-NLMS) while it resolves the high steady-state error value and the complexity issues in the regular affine projection algorithm. The fast convergence of the proposed algorithm means that a shorter training sequence inside each data block is required, which in turn improves the effective bit rate. This fast convergence is more pronounced when the pilot vectors are highly correlated. We incorporated the set-membership filtering framework into our study to reduce the computational complexity of the algorithm and preserve energy in the WSNs. Other studies have shown the superiority of the adaptive filtering algorithms, and in particular the NLMS algorithm, over other alternatives in various signal processing areas in WSNs, therefore, our proposed algorithm is a powerful substitute for a variety of algorithms. In other words, the implementation of various signal processing algorithms for different purposes can be replaced with the implementation of the proposed multipurpose algorithm. In this paper, we combine the results of our previous studies and prove the convergence of the algorithm. Furthermore, the steady-state analysis in the output mean square error is presented for two cases of pilot signals, and in the conducted simulations, the MSE performance of the algorithm is compared with the regular affine projection algorithm and the SM-NLMS algorithm. [ABSTRACT FROM AUTHOR]

Synchronization of bio-inspired pulse-coupled oscillators is receiving increased attention due to its wide applications in sensor networks and wireless communications. However, most existing results are obtained in the absence of malicious attacks. Given the distributed and unattended nature of wireless sensor networks, it is imperative to enhance the resilience of pulse based synchronization against malicious attacks. To achieve this goal, we propose a new pulse based interaction mechanism to improve the resilience of pulse based synchronization. We rigorously characterize the condition for mounting stealthy attacks under the proposed pulse based interaction mechanism and prove analytically that synchronization of legitimate oscillators can be achieved in the presence of multiple stealthy attackers even when the initial phases are unrestricted, i.e., randomly distributed in the entire oscillation period. This is in distinct difference from most existing attack-resilient synchronization algorithms (including the seminal paper from L. Lamport and P. M. Melliar-Smith, “Synchronizing Clocks in the Presence of Faults,” J. ACM, vol. 32, no. 1, pp. 52–78, 1985), which require a priori (almost) synchronization among legitimate nodes. Numerical simulations are given to confirm the theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, we propose to fit a sparse logistic regression model by a weakly convex regularized nonconvex optimization problem. The idea is based on the finding that a weakly convex function as an approximation of the $\ell _0$ pseudo norm is able to better induce sparsity than the commonly used $\ell _1$ norm. For a class of weakly convex sparsity inducing functions, we prove the nonconvexity of the corresponding problem and study its local optimality conditions and the choice of the regularization parameter. Despite the nonconvexity, a method based on proximal gradient descent is used to solve the general weakly convex sparse logistic regression, and its convergence behavior is studied theoretically. Then, the general framework is applied to a specific weakly convex function, and a local optimality condition and a bound on the logistic loss at a local optimum are provided. The solution method is instantiated in this case as an iterative firm-shrinkage algorithm, and a Nesterov acceleration is used with a convergence guarantee. Its effectiveness is demonstrated in numerical experiments by both randomly generated and real datasets. [ABSTRACT FROM AUTHOR]

A fundamental problem in multimodal signal processing is to quantify relations between two different signals with respect to a certain phenomenon. In this paper, we address this problem from a kernel-based perspective and propose a measure that is based on affinity kernels constructed separately in each modality. This measure is motivated from both a kernel density estimation point of view of predicting the signal in one modality based on the other, as well as from a statistical model, which implies that high values of the proposed measure are expected when signals highly correspond to each other. Considering an online setting, we propose an efficient algorithm for the sequential update of the proposed measure, and demonstrate its application to eye-fixation prediction in audio-visual recordings. The goal is to predict locations within a video recording at which people gaze when watching the video. As studies in psychology imply, people tend to gaze at the location of the audio source, so that their prediction becomes equivalent to locating the audio source within the video. Therefore, we propose to predict eye-fixations as regions within the video with the highest correspondence to the audio signal, thereby demonstrating the improved performance of the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a low complexity n-dimensional (nD) FastICA algorithm and architecture by introducing the concept of coordinate rotation where n\geq 2. The proposed algorithm can merge the two key steps of conventional FastICA—preprocessing and update and is therefore capable of reducing the hardware complexity of the conventional FastICA significantly as demonstrated in this paper. Hardware implementation can further be simplified due to the recursive nature of the proposed algorithm where the same 2D hardware module can be used as the fundamental core to implement nD architecture. Together with the algorithm formulation, its functionality is also validated and hardware complexity is analyzed and compared with the conventional nD FastICA. [ABSTRACT FROM PUBLISHER]

In this work, we apply deep learning to estimate the direction-of-arrival (DoA) of multiple narrowband signals with a uniform linear array in a coherent environment. First, the logarithmic eigenvalue-based classification network (LogECNet) is introduced to enhance signal number detection accuracy in challenging scenarios, such as the low signal-to-noise (SNR) regime and limited snapshots. Next, a multi-label classification model called the root-spectrum network (RSNet) is devised to estimate the DoAs using the signal number inferred by LogECNet. In the proposed architecture, the full-row Toeplitz matrices reconstruction (FTMR), which exploits all rows of the signal covariance matrix (SCM), is combined with LogECNet and RSNet to inversely map the SCM to the numerical DoAs in the coherent scenario. It is shown that the eigenvalues factorized from the FTMR output matrix become more robust sources for signal enumeration than those of the forward/backward spatial smoothing (FBSS) algorithm. Furthermore, the logarithmic scaling of the eigenvalues of the FTMR results in LogECNet outperforming other detectors. The simulation results show our proposed method not only improves the signal number detection and angular estimation performance, but also achieves the complexity reduction with respect to the prior schemes. [ABSTRACT FROM AUTHOR]

Modern information processing relies on the axiom that high-dimensional data lie near low-dimensional geometric structures. This paper revisits the problem of data-driven learning of these geometric structures and puts forth two new nonlinear geometric models for data describing “related” objects/phenomena. The first one of these models straddles the two extremes of the subspace model and the union-of-subspaces model, and is termed the metric-constrained union-of-subspaces (MC-UoS) model. The second one of these models—suited for data drawn from a mixture of nonlinear manifolds—generalizes the kernel subspace model, and is termed the metric-constrained kernel union-of-subspaces (MC-KUoS) model. The main contributions of this paper in this regard include the following. First, it motivates and formalizes the problems of MC-UoS and MC-KUoS learning. Second, it presents algorithms that efficiently learn an MC-UoS or an MC-KUoS underlying data of interest. Third, it extends these algorithms to the case when parts of the data are missing. Last, but not least, it reports the outcomes of a series of numerical experiments involving both synthetic and real data that demonstrate the superiority of the proposed geometric models and learning algorithms over existing approaches in the literature. These experiments also help clarify the connections between this work and the literature on (subspace and kernel k-means) clustering. [ABSTRACT FROM AUTHOR]

In Part I of the study of the tensor deflation for CANDECOMP/PARAFAC, we have shown that the rank-1 tensor deflation is applicable under some conditions. Part II of the study presents several initialization algorithms suitable for the algorithm proposed in Part I. In addition, Part II contains an algorithm for the case when one or more factor matrices in the estimated model is constrained to be orthogonal. Finally, Part II provides an error analysis of the tensor deflation algorithm, which shows that there is a marginal loss of accuracy of the deflation algorithm compared to the ordinary CP decomposition. [ABSTRACT FROM PUBLISHER]

We correct errors in step 6 of Algorithm 1 given in the paper ‘'Sparse-Group Lasso for Graph Learning from Multi-Attribute Data.’' Other parts of the paper including numerical results are unchanged as the algorithm was implemented correctly. [ABSTRACT FROM AUTHOR]

Time synchronization is crucial for wireless sensor networks (WSNs) since it maintains data consistency, coordination, and enables other fundamental operations. Over the last decade, a number of powerful time synchronization algorithms have been proposed for clock accuracy optimization by using signal processing techniques. However, most of these algorithms attempt to estimate the clock phase offset and skew and do not correct the node's local clock at every timing packet exchange before the clock parameters are jointly estimated, which reduces the network synchronization accuracy during the estimation of clock parameters and limits the applicability of the synchronization algorithms for some practical sensor networks where a global time scale is consistently required. Motivated by this, this paper investigates the synchronization scheme of having inactive nodes overhearing the pairwise sender-receiver time synchronization based on acknowledgement with clock correction at every synchronization. Assuming exponential random delays, the maximum likelihood estimators (MLEs) of clock skew and the corresponding approximate Cramer-Rao lower bounds (CRLBs) for active and overhearing nodes are derived. In addition, we also consider a linear synchronization model that the clock skew is directly estimated. Simulation results verify the efficiency of the clock skew estimators. [ABSTRACT FROM PUBLISHER]

This paper studies the stochastic behavior of the signed variants of the LMS algorithm for a system identification framework when the input signal is a cyclostationary white Gaussian process. Three algorithms are studied: the signed regressor, the signed error, and the sign–sign algorithms. The input cyclostationary signal is modeled by a white Gaussian random process with periodically time-varying power. The system parameters vary according to a random-walk. Mathematical models are derived for the mean and mean-square-deviation behavior of the adaptive weights with the input cyclostationarity. These models are used to derive new results concerning the performance of the algorithms. Some of these results are surprising. Monte Carlo simulations of the three algorithms provide strong support for the theory. [ABSTRACT FROM PUBLISHER]

Particle filters are broadly used to approximate posterior distributions of hidden states in state-space models by means of sets of weighted particles. While the convergence of the filter is guaranteed when the number of particles tends to infinity, the quality of the approximation is usually unknown but strongly dependent on the number of particles. In this paper, we propose a novel method for assessing the convergence of particle filters in an online manner, as well as a simple scheme for the online adaptation of the number of particles based on the convergence assessment. The method is based on a sequential comparison between the actual observations and their predictive probability distributions approximated by the filter. We provide a rigorous theoretical analysis of the proposed methodology and, as an example of its practical use, we present simulations of a simple algorithm for the dynamic and online adaptation of the number of particles during the operation of a particle filter on a stochastic version of the Lorenz 63 system. [ABSTRACT FROM PUBLISHER]