Search

This paper studies the stochastic behavior of the LMS and NLMS algorithms in a system identification framework for a cyclostationary white input without assuming a Gaussian distribution for the input. The input cyclostationary signal is modeled by a white random process with periodically time-varying power. The system parameters vary according to a random-walk. Mathematical models are derived for the mean and mean-square-deviation behavior of the adaptive weights as a function of the input cyclostationarity. Analytical models are first derived for the LMS and NLMS algorithms for cyclostationary white inputs. These models show the dependence of the two algorithms upon the kurtosis of the input. Significant differences are found between the behaviors of the two algorithms when the analysis is applied to non-Gaussian cases. Monte Carlo simulations provide strong support for the theory. [ABSTRACT FROM AUTHOR]

The low-rank matrix completion problem is fundamental to a number of tasks in data mining, machine learning, and signal processing. This paper considers the problem of adaptive matrix completion in time-varying scenarios. Given a sequence of incomplete and noise-corrupted matrices, the goal is to recover and track the underlying low rank matrices. Motivated from the classical least-mean square (LMS) algorithms for adaptive filtering, three LMS-like algorithms are proposed for estimating and tracking low-rank matrices. Performance of the proposed algorithms is provided in form of nonasymptotic bounds on the tracking mean-square error. Tracking performance of the algorithms is also studied via detailed simulations over real-world datasets. [ABSTRACT FROM PUBLISHER]

Stochastic approximation techniques play an important role in solving many problems encountered in machine learning or adaptive signal processing. In these contexts, the statistics of the data are often unknown a priori or their direct computation is too intensive, and they have thus to be estimated online from the observed signals. For batch optimization of an objective function being the sum of a data fidelity term and a penalization (e.g., a sparsity promoting function), Majorize-Minimize (MM) methods have recently attracted much interest since they are fast, highly flexible, and effective in ensuring convergence. The goal of this paper is to show how these methods can be successfully extended to the case when the data fidelity term corresponds to a least squares criterion and the cost function is replaced by a sequence of stochastic approximations of it. In this context, we propose an online version of an MM subspace algorithm and we study its convergence by using suitable probabilistic tools. Simulation results illustrate the good practical performance of the proposed algorithm associated with a memory gradient subspace, when applied to both nonadaptive and adaptive filter identification problems. [ABSTRACT FROM PUBLISHER]

The success of compressed sensing relies essentially on the ability to efficiently find an approximately sparse solution to an under-determined linear system. In this paper, we developed an efficient algorithm for the sparsity promoting \ell^1-regularized least squares problem by coupling the primal dual active set strategy with a continuation technique (on the regularization parameter). In the active set strategy, we first determine the active set from primal and dual variables, and then update the primal and dual variables by solving a low-dimensional least square problem on the active set, which makes the algorithm very efficient. The continuation technique globalizes the convergence of the algorithm, with provable global convergence under restricted isometry property (RIP). Further, we adopt two alternative methods, i.e., a modified discrepancy principle and a Bayesian information criterion, to choose the regularization parameter automatically. Numerical experiments indicate that our algorithm is very competitive with state-of-the-art algorithms in terms of accuracy and efficiency, without a priori information about the regularization parameter. [ABSTRACT FROM AUTHOR]

Sparse signal restoration is usually formulated as the minimization of a quadratic cost function \Vert \mbi y - \mbi A \mbi x \Vert2^{2} where \mbi A is a dictionary and \mbi x is an unknown sparse vector. It is well-known that imposing an \ell 0 constraint leads to an NP-hard minimization problem. The convex relaxation approach has received considerable attention, where the \ell 0-norm is replaced by the \ell 1-norm. Among the many effective \ell 1 solvers, the homotopy algorithm minimizes \Vert \mbi y - \mbi A \mbi x \Vert2^{2}+\lambda \Vert \mbi x \Vert 1 with respect to \mbi x for a continuum of \lambda ’s. It is inspired by the piecewise regularity of the \ell 1-regularization path, also referred to as the homotopy path. In this paper, we address the minimization problem \Vert \mbi y - \mbi A \mbi x \Vert2^{2}+\lambda \Vert \mbi x \Vert 0 for a continuum of \lambda ’s and propose two heuristic search algorithms for \ell 0-homotopy. Continuation Single Best Replacement is a forward–backward greedy strategy extending the Single Best Replacement algorithm, previously proposed for \ell 0-minimization at a given \lambda . The adaptive search of the \lambda -values is inspired by \ell 1-homotopy. \ell 0 Regularization Path Descent is a more complex algorithm exploiting the structural properties of the \ell 0-regularization path, which is piecewise constant with respect to \lambda . Both algorithms are empirically evaluated for difficult inverse problems involving ill-conditioned dictionaries. Finally, we show that they can be easily coupled with usual methods of model order selection. [ABSTRACT FROM PUBLISHER]

This paper considers the average-consensus problem in a network with arbitrary (but finite) communication delays. A novel Distributed-Averaging (DA) algorithm is presented and shown to achieve average-consensus if at any time t there exists a finite time interval [t, Tt] over which each node can communicate (via a time-respecting path) with all other nodes. For consensus variables with dimensions on the order of the network size, the DA algorithm requires an order of magnitude less data to be communicated and stored at each node as compared to an idealized algorithm that floods the initial data. In the companion paper refid="ref29"/ , practical applications of the DA algorithm are provided along with numerical examples. [ABSTRACT FROM PUBLISHER]

Constrained least-squares design and constrained Chebyshev design of one- and two-dimensional nonlinear-phase FIR filters with prescribed phase error are considered in this paper by a unified semi-infinite positive-definite quadratic programming approach. In order to obtain unique optimal solutions, we propose to impose constraints on the complex approximation error and the phase error. By introducing a sigmoid phase-error constraint bound function, the group-delay error can be greatly reduced. A Goldfarb-Idnani based algorithm is presented to solve the semi-infinite positive-definite quadratic program resulting from the constrained least-squares design problem, and then applied after some modifications to the constrained Chebyshev design problem, which is proved in this paper to be equivalent also to a semi-infinite positive-definite quadratic program. Through design examples, the proposed method is compared with several existing methods. Simulation results demonstrate the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

This paper presents an analysis of the steady-state mean-square error (MSE) of the set-membership normalized least-mean square (SM-NLMS) algorithm with relaxation and regularization parameters. These parameters are introduced for the purpose of deriving in a unified way the steady-state MSE performances of the ϵ-normalized least mean square (ϵ-NLMS) algorithm and a special case of the adaptive parallel subgradient projection (PSP) algorithm. The approach of the paper is to employ the energy conservation relation as a starting point of our analysis. This relation enables us to avoid the transient analysis of the SM-NLMS algorithm, which is in general hard due to the nonlinearity of the SM-NLMS algorithm. As a result, a few nonlinear equations whose solutions are theoretical steady-state MSEs are derived, where two types of reasonable assumptions are introduced to overcome the nonlinearity of the SM-NLMS algorithm. Our results are generalizations of well-known results of the steady-state MSE of the ϵ-NLMS. Extensive simulations show the close agreement between our theories and experiments. [ABSTRACT FROM AUTHOR]

This paper presents coefficient filtering techniques in the least mean squares (LMS) algorithm to improve adaptive predictor tracking performance for time-varying chirped signals. The example application used in this paper is an electronic support measure (ESM) receiver for detecting radar chirped pulses. The leakage LMS, momentum LMS, and the proposed future-state coefficient (FC-LMS) filtering algorithms have been studied. The leakage LMS algorithm has the ability to remove the memory effect of the initial converged time-varying frequency of the chirped signal, thus Improving the radar pulse detection performance. The momentum LMS is able to search for the time-varying optimum weight solution more efficiently, and the FC-LMS uses a parallel technique to retain the LMS throughput while being able, to show a better tracking performance for chirped' signals compared with the standard LMS algorithm. [ABSTRACT FROM AUTHOR]

Unit-modulus least squares (ULS) problems arise in many applications, including phase-only beamforming, sensor network localization, synchronization, phase retrieval, and radar code design. ULS formulations can always be recast as unit-modulus quadratic programs, to which semidefinite relaxation (SDR) can be applied, and is often the state-of-the-art approach. However, SDR lifts the problem dimension (i.e., the number of variables) from $N$ to $N^2$, which drastically increases the memory burden and computational cost when the problem size is already large—e.g., when designing phase-only beamformer weights for massive multiple-input-multiple-output systems. This paper focuses on scalable first-order algorithms for the ULS problem and some of its variants. It advocates using simple gradient projection (GP) as a starting point for solving the ULS problem, establishes global convergence of GP to a Karush–Kuhn–Tucker point for this NP-hard problem, and bounds its iteration complexity. Then it proposes ULS extensions tailored to reflect practical beamformer design objectives, bringing in and exploiting new degrees of freedom to improve the beampattern designs. Simple variants of GP are proposed to deal with these extended ULS problems. Simulations are used to showcase the effectiveness of the proposed algorithms in both the plain ULS problem and in the context of phase-only beamforming. [ABSTRACT FROM PUBLISHER]

A precise model of the array response is required to maintain the performance of direction-of-arrival (DOA) estimation. When modeling errors are present or the sensor environment is time-varying, autocalibration becomes necessary. In this paper, the problem of phase autocalibration for uniform rectangular array (URA) geometries is considered. For the case with a single source, a simple and robust least-squares algorithm for joint 2-D DOA estimation and phase calibration is presented. When performing phase autocalibration with a URA, the phase and DOA parameters cannot be identified together without ambiguity. This problem is discussed and a suitable remedy is suggested. An approximate Cramér-Rao bound and analytical expressions for the mean squared error performance of the proposed estimator are presented. The proposed algorithm for phase autocalibration is extended for the case with multiple sources. The results are evaluated using a representative body of simulations. [ABSTRACT FROM PUBLISHER]

Dynamic system modeling plays a crucial role in the development of techniques for stationary and nonstationary signal processing. Due to the inherent physical characteristics of systems under investigation, nonnegativity is a desired constraint that can usually be imposed on the parameters to estimate. In this paper, we propose a general method for system identification under nonnegativity constraints. We derive the so-called nonnegative least-mean-square algorithm (NNLMS) based on stochastic gradient descent, and we analyze its convergence. Experiments are conducted to illustrate the performance of this approach and consistency with the analysis. [ABSTRACT FROM AUTHOR]

Using the ℓ1-norm to regularize the least-squares criterion, the batch least-absolute shrinkage and selection operator (Lasso) has well-documented merits for estimating sparse signals of interest emerging in various applications where observations adhere to parsimonious linear regression models. To cope with high complexity, increasing memory requirements, and lack of tracking capability that batch Lasso estimators face when processing observations sequentially, the present paper develops a novel time-weighted Lasso (TWL) approach. Performance analysis reveals that TWL cannot estimate consistently the desired signal support without compromising rate of convergence. This motivates the development of a time- and norm-weighted Lasso (TNWL) scheme with ℓ1-norm weights obtained from the recursive least-squares (RLS) algorithm. The resultant algorithm consistently estimates the support of sparse signals without reducing the convergence rate. To cope with sparsity-aware recursive real-time processing, novel adaptive algorithms are also developed to enable online coordinate descent solvers of TWL and TNWL that provably converge to the true sparse signal in the time-invariant case. Simulated tests compare competing alternatives and corroborate the performance of the novel algorithms in estimating time-invariant signals, and tracking time-varying signals under sparsity constraints. [ABSTRACT FROM AUTHOR]

The S transform is useful in time-frequency analysis. Many inverse S transform algorithms have been proposed with different filtering properties in the time-frequency spectrum. In this paper, the transformation matrices of the S transform and two novel least square inverse algorithms are proposed. The first one minimizes the global mean square error of the entire time-frequency spectrum, and the second one considers only the specific interesting time-frequency regions and is more flexible. The proposed inverse algorithms can provide more stable and better performance than the existing ones. [ABSTRACT FROM AUTHOR]

In this paper, we develop a novel methodology for minimizing a class of nonconvex (concave on the non-negative orthant) functions for solving an underdetermined linear system of equations As = x when the solution vector is known a priori to be sparse. The proposed technique is based on locally replacing the original objective function by a quadratic convex function which is easily minimized. The resulting algorithm is iterative and is absolutely converging to a fixed point of the original objective function. For a certain selection of convex objective functions, the class of algorithms called iterative reweighted least squares (IRLS) is shown to be a special case of the proposed methodology. Thus, the proposed algorithms are a generalization and unification of the previous methods. In addition, we also propose a new class of algorithms with better convergence properties compared to the regular IRLS algorithms and, hence, can be considered as enhancements to these algorithms. Since the original objective functions are nonconvex, the proposed algorithm is susceptible to convergence to a local minimum. To alleviate this difficulty, we propose a random perturbation technique that enhances the performance of the proposed algorithm. The numerical results show that the proposed algorithms outperform some of the well-known algorithms that are usually utilized for solving the same problem. [ABSTRACT FROM AUTHOR]

This paper proposes a general framework to effectively estimate the unknown timing and channel parameters, as well as design efficient timing resynchronization algorithms for asynchronous amplify-and-forward (AF) cooperative communication systems. In order to obtain reliable timing and channel parameters, a least squares (LS) estimator is proposed for initial estimation and an iterative maximum-likelihood (ML) estimator is derived to refine the LS estimates. Furthermore, a timing and channel uncertainty analysis based on the Cramér-Rao bounds (CRB) is presented to provide insights into the system uncertainties resulted from estimation. Using the parameter estimates and uncertainty information in our analysis, timing resynchronization algorithms that are robust to estimation errors are designed jointly at the relays and the destination. The proposed framework is developed for different AF systems with varying degrees of timing misalignment and channel uncertainties and is numerically shown to provide excellent performances that approach the synchronized case with perfect channel information. [ABSTRACT FROM AUTHOR]

Iteratively reweighted least-squares (IRLS) algorithms have been successfully used in compressive sensing to reconstruct sparse signals from incomplete linear measurements taken in nonsparse domains. The underlying optimization problem corresponds to finding the vector that solves the ℓp, minimization while explaining the measurements, and IRLS allows to easily control the used value of p, with effect on the number of required measurements. In this paper, we propose a weighting strategy in the reconstruction method based on IRLS in order to add prior information on the support of the sparse domain. Our simulation results show that the use of prior knowledge about positions of at least some of the nonzero coefficients in the sparse domain leads to a reduction in the number of linear measurements required for unambiguous reconstruction. This reduction occurs for all values of p, so that a further reduction can be achieved by decreasing p and using prior information. The proposed weighting scheme also reduces the computational complexity with respect to the IRLS with no prior information, both in terms of number of iterations and computation time. [ABSTRACT FROM AUTHOR]

In this paper, we present a powerful technique for the blind extraction of direct-sequence code-division multiple access (DS-CDMA) signals from convolutive mixtures received by an antenna array. The technique is based on a generalization of the canonical or parallel factor decomposition (CANDECOMP/PARAFAC) in multilinear algebra. We present a bound on the number of users under which blind separation and deconvolution is guaranteed. The solution is computed by means of an alternating least squares (ALS) algorithm. The excellent performance is illustrated by means of a number of simulations. We include an explicit expression of the Cramér-Rao bound (CRB) of the transmitted symbols. [ABSTRACT FROM AUTHOR]

In this paper, we consider sequential regression of individual sequences under the square-error loss. We focus on the class of switching linear predictors that can segment a given individual sequence into an arbitrary number of blocks within each of which a fixed linear regressor is applied. Using a competitive algorithm framework, we construct sequential algorithms that are competitive with the best linear regression algorithms for any segmenting of the data as well as the best partitioning of the data into any fixed number of segments, where both the segmenting of the data and the linear predictors within each segment can be tuned to the underlying individual sequence. The algorithms do not require knowledge of the data length or the number of piecewise linear segments used by the members of the competing class, yet can achieve the performance of the best member that can choose both the partitioning of the sequence as well as the best regressor within each segment. We use a transition diagram (F. M. J. Willems, 1996) to compete with an exponential number of algorithms in the class, using complexity that is linear in the data length. The regret with respect to the best member is O(ln(n)) per transition for not knowing the best transition times and O(ln(n)) for not knowing the best regressor within each segment, where n is the data length. We construct lower bounds on the performance of any sequential algorithm, demonstrating a form of mm-max optimality under certain settings. We also consider the case where the members are restricted to choose the best algorithm in each segment from a finite collection of candidate algorithms. Performance on synthetic and real data are given along with a Matlab implementation of the universal switching linear predictor. [ABSTRACT FROM AUTHOR]

This paper proposes a new fast recursive total least squares (N-RTLS) algorithm to recursively compute the TLS solution for adaptive finite impulse response (FIR) filtering. The N-RTLS algorithm is based on the minimization of the constrained Rayleigh quotient (c-RQ) in which the last entry of the parameter vector is constrained to the negative one. As analysis results on the convergence of the proposed algorithm, we study the properties of the stationary points of the c-RQ. The high computational efficiency of the new algorithm depends on the efficient computation of the fast gain vector (FGV) and the adaptation of the c-RQ. Since the last entry of the parameter vector in the c-RQ has been fixed as the negative one, a minimum point of the c-RQ is searched only along the input data vector, and a more efficient N-RTLS algorithm is obtained by using the FGV. As compared with Davila's RTLS algorithms, the N-RTLS algorithm saves the GM number of multiplies, divides, and square roots (MADs). The global convergence of the new algorithm is studied by LaSalle's invariance principle. The performances of the relevant algorithms are compared via simulations, and the long-term numerical stability of the N-RTLS algorithm is verified. [ABSTRACT FROM AUTHOR]

High-resolution images are often desired but made impossible because of hardware limitations. For the high-resolution model proposed by Bose and Boo, the iterative wavelet-based algorithm has been shown to perform better than the traditional least square method when the resolution ratio M is two and four. In this paper, we discuss the minimally supported biorthogonal wavelet system that comes from the mathematical model by Bose and Boo and propose a wavelet-based algorithm for arbitrary resolution ratio M ≥ 2. The numerical results indicate that the algorithm based on our biorthogonal wavelet system performs better in high-resolution image reconstruction than the wavelet-based algorithm in the literature, as well as the common-used least square method. [ABSTRACT FROM AUTHOR]

In this paper, we present a new design method of infinite impulse response (IIR) digital filters with quasi-equiripple absolute error in the complex domain. This method is based on solving a least squares solution iteratively. At each iteration, the desired response for the least squares approximation is transformed to have equiripple error. This algorithm is efficient because there is no need for any initial value or complex optimization algorithm. By this method, a quasi-equiripple solution is obtained very quickly with less computational complexity. Moreover, by multiplying an arbitrary weighting function on the desired responses of passband and stopband, respectively, the error at the passband and stopband can be controlled. Finally, we show some examples to validate the proposed method. [ABSTRACT FROM AUTHOR]

The area of sparse approximation of signals is drawing tremendous attention in recent years. Typically, sparse solutions of underdetermined linear systems of equations are required. Such solutions are often achieved by minimizing an l1 penalized least squares functional. Various iterative-shrinkage algorithms have recently been developed and are quite effective for handling these problems, often surpassing traditional optimization techniques. In this paper, we suggest a new iterative multilevel approach that reduces the computational cost of existing solvers for these inverse problems. Our method takes advantage of the typically sparse representation of the signal, and at each iteration it adaptively creates and processes a hierarchy of lower-dimensional problems employing well-known iterated shrinkage methods. Analytical observations suggest, and numerical results confirm, that this new approach may significantly enhance the performance of existing iterative shrinkage algorithms in cases where the matrix is given explicitly. [ABSTRACT FROM PUBLISHER]

In signal processing, data analysis and scientific computing, one often encounters the problem of decomposing a tensor into a sum of contributions. To solve such problems, both the search direction and the step size are two crucial elements in numerical algorithms, such as alternating least squares algorithm (ALS). Owing to the nonlinearity of the problem, the often used linear search direction is not always powerful enough. In this paper, we propose two higher-order search directions. The first one, geometric search direction, is constructed via a combination of two successive linear directions. The second one, algebraic search direction, is constructed via a quadratic approximation of three successive iterates. Then, in an enhanced line search along these directions, the optimal complex step size contains two arguments: modulus and phase. A current strategy is ELSCS that finds these two arguments alternately. So it may suffer from a local optimum. We broach a direct method, which determines these two arguments simultaneously, so as to obtain the global optimum. Finally, numerical comparisons on various search direction and step size schemes are reported in the context of blind separation-equalization of convolutive DS-CDMA mixtures. The results show that the new search directions have greatly improve the efficiency of ALS and the new step size strategy is competitive. [ABSTRACT FROM AUTHOR]

Coping with outliers contaminating dynamical processes is of major importance in various applications because mismatches from nominal models are not uncommon in practice. In this context, the present paper develops novel fixed-lag and fixed-interval smoothing algorithms that are robust to outliers simultaneously present in the measurements and in the state dynamics. Outliers are handled through auxiliary unknown variables that are jointly estimated along with the state based on the least-squares criterion that is regularized with the \ell1-norm of the outliers in order to effect sparsity control. The resultant iterative estimators rely on coordinate descent and the alternating direction method of multipliers, are expressed in closed form per iteration, and are provably convergent. Additional attractive features of the novel doubly robust smoother include: i) ability to handle both types of outliers; ii) universality to unknown nominal noise and outlier distributions; iii) flexibility to encompass maximum a posteriori optimal estimators with reliable performance under nominal conditions; and iv) improved performance relative to competing alternatives at comparable complexity, as corroborated via simulated tests. [ABSTRACT FROM PUBLISHER]

In this paper, we provide a thorough stability analysis of two well known adaptive algorithms for equalization based on a novel least squares reference model that allows to treat the equalizer problem equivalently as system identification problem. While not surprising the adaptive minimum mean-square error (MMSE) equalizer algorithm behaves l2–stable for a wide range of step-sizes, the even older zero-forcing (ZF) algorithm however behaves very differently. We prove that the ZF algorithm generally does not belong to the class of robust algorithms but can be convergent in the mean square sense. We furthermore provide conditions on the upper step-size bound to guarantee such mean squares convergence. We specifically show how noise variance of added channel noise and the channel impulse response influences this bound. Simulation examples validate our findings. [ABSTRACT FROM AUTHOR]

Channel order estimation is a critical task for blind system identification. The performance of the blind system identification algorithms depends on the accuracy and robustness of the channel order estimation. In this paper, a new effective channel order estimation algorithm with high accuracy and robustness is proposed for single-input multi-output (SIMO) systems. The proposed algorithm is guaranteed to find the true channel order for the noise-free case and it performs significantly better than the alternative algorithms for noisy observations. This algorithm shows a consistent performance when the number of observations, channels and channel order are changed. The proposed algorithm is integrated with the least squares smoothing (LSS) algorithm for blind identification of the channel coefficients. Comparisons are done with a variety of different algorithms including linear prediction (LP) based methods. It is shown that significant gain can be obtained compared to the alternative approaches in effective channel order estimation. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a reduced-rank space-time adaptive processing (STAP) technique for airborne phased array radar applications. The proposed STAP method performs dimensionality reduction by using a reduced-rank switched joint interpolation, decimation and filtering algorithm (RR-SJIDF). In this scheme, a multiple-processing-branch (MPB) framework, which contains a set of jointly optimized interpolation, decimation and filtering units, is proposed to adaptively process the observations and suppress jammers and clutter. The output is switched to the branch with the best performance according to the minimum variance criterion. In order to design the decimation unit, we present an optimal decimation scheme and a low-complexity decimation scheme. We also develop two adaptive implementations for the proposed scheme, one based on a recursive least squares (RLS) algorithm and the other on a constrained conjugate gradient (CCG) algorithm. The proposed adaptive algorithms are tested with simulated radar data. The simulation results show that the proposed RR-SJIDF STAP schemes with both the RLS and the CCG algorithms converge at a very fast speed and provide a considerable SINR improvement over the state-of-the-art reduced-rank schemes. [ABSTRACT FROM AUTHOR]

In this paper, we propose an approach to the transient and steady-state analysis of the affine combination of one fast and one slow adaptive filters. The theoretical models are based on expressions for the excess mean-square error (EMSE) and cross-EMSE of the component filters, which allows their application to different combinations of algorithms, such as least mean-squares (LMS), normalized LMS (NLMS), and constant modulus algorithm (CMA), considering white or colored inputs and stationary or nonstationary environments. Since the desired universal behavior of the combination depends on the correct estimation of the mixing parameter at every instant, its adaptation is also taken into account in the transient analysis. Furthermore, we propose normalized algorithms for the adaptation of the mixing parameter that exhibit good performance. Good agreement between analysis and simulation results is always observed. [ABSTRACT FROM AUTHOR]

This paper presents a proportionate normalized least-mean-square (PNLMS) algorithm using individual activation factors for each adaptive filter coefficient, instead of a global activation factor as in the standard PNLMS algorithm. The proposed individual activation factors, determined in terms of the corresponding adaptive filter coefficients, are recursively updated. This approach leads to a better distribution of the adaptation energy over the filter coefficients than the standard PNLMS does. Thereby, for impulse responses exhibiting high sparseness, the proposed algorithm achieves faster convergence, outperforming both the PNLMS and improved PNLMS (IPNLMS) algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes a novel adaptive filtering scheme named the Krylov-proportionate normalized least-mean-square (KPNLMS) algorithm. KPNLMS exploits the benefits (i.e., fast convergence for sparse unknown systems) of the proportionate NLMS algorithm, but its applications are not limited to sparse unknown systems. A set of orthonormal basis vectors is generated from a certain Krylov sequence. It is proven that the unknown system is sparse with respect to the basis vectors in case of fairly uncorrelated input data. Different adaptation gain is allocated to a coefficient of each basis vector, and the gain is roughly proportional to the absolute value of the corresponding coefficient of the current estimate. KPNLMS enjoys i) fast convergence, ii) linear complexity per iteration, and iii) no use of any a priori information. Numerical examples demonstrate significant advantages of the proposed scheme over the reduced-rank method based on the multistage Wiener filter (MWF) and the transform-domain adaptive filter (TDAF) both in noisy and silent situations. [ABSTRACT FROM AUTHOR]

Bearings-only tracking (BOT) and Doppler-bearing tracking (DBT) perform target motion analysis from measure- ments of bearings, Doppler and bearings, respectively. The state equations are simple for both the target and observer in the rectangular coordinates. Hence, BOT and DBT mostly work in that coordinate. By developing the trigonometric relations between target and observer postions over time, this paper first derives the pseudolinear equations for direct range and bearing estimation. It then provides a generalized total least squares (GTLS), and a maximum likelihood solution. Polar tracking has two advantages over rectangular. First, for applications that require range and bearing, polar output avoids a coordinate conversion at every output instant. Second, polar equations often have a smaller bias than rectangular. Consequently, polar estimates can give a closer (to the optimum) initialization for iterative algorithms. Simulation results have validated the theoretical development, confirming the near optimality of the GTLS and maximum likelihood solutions. [ABSTRACT FROM AUTHOR]

This paper deals with the problem of adaptive reconstruction and identification of nonstationary AR processes with randomly missing observations. Existent methods use a direct realization of the filter. Therefore, the estimated parameters may not correspond to a stable all-pole filter. In addition, when the probability of missing a sample is high, existent methods may converge slowly or even fail to converge. We propose, at our knowledge, the first algorithm based on the lattice structure for online processing of signals with missing samples. It is an extension of the RLSL algorithm to the case of missing observations, using a Kalman filter for the prediction of missing samples. The estimated parameters guarantee the stability of the corresponding all-pole filter. In addition it is robust to high probabilities of missing a sample. It offers a fast parameter tracking even for high probabilities of missing a sample. It is compared to the Kalman pseudolinear RLS algorithm, an already proposed algorithm using a direct realization of the filter. The proposed algorithm shows better performance in reconstruction of audio signals. [ABSTRACT FROM PUBLISHER]

This paper proposes a multistage decomposition for blind adaptive parameter estimation in the Krylov subspace with the code-constrained constant modulus (CCM) design criterion. Based on constrained optimization of the constant modulus cost function and utilizing the Lanczos algorithm and Arnoldi-like iterations, a multistage decomposition is developed for blind parameter estimation. A family of computationally efficient blind adaptive reduced-rank stochastic gradient (SG) and recursive least squares (RLS) type algorithms along with an automatic rank selection procedure are also devised and evaluated against existing methods. An analysis of the convergence properties of the method is carried out and convergence conditions for the reduced-rank adaptive algorithms are established. Simulation results consider the application of the proposed techniques to the suppression of multiaccess and intersymbol interference in DS-CDMA systems. [ABSTRACT FROM AUTHOR]

While the theory of estimation of monocomponent polynomial phase signals is well established, the theoretical and methodical treatment of multicomponent polynomial phase signals (mc-PPSs) is limited. In this paper, we investigate several aspects of parameter estimation for mc-PPSs and derive the Cramér-Rao bound. We show the limits of existing techniques and then propose a nonlinear least squares (NLS) approach. We also motivate the use the Nelder-Mead simplex algorithm for minimizing the nonlinear cost function. The slight increase in computational complexity is a tradeoff for improved mean square error performance, which is evidenced by simulation results. [ABSTRACT FROM AUTHOR]

The adaptive parallel subgradient projection (PSP) algorithm was proposed in 2002 as a set-theoretic adaptive filtering algorithm providing fast and stable convergence, robustness against noise, and low computational complexity by using weighted parallel projections onto multiple time-varying closed half-spaces. In this paper, we present a novel weighting technique named pairwise optimal weight realization (POWER) for further acceleration of the adaptive PSP algorithm. A simple closed-form formula is derived to compute the projection onto the intersection of two closed half-spaces defined by a triplet of vectors. Using the formula inductively, the proposed weighting technique realizes a good direction of update. The resulting weights turn out to be pairwise optimal in a certain sense. The proposed algorithm has the inherently parallel structure composed of q primitive functions, hence its total computational complexity O(qrN) is reduced to O(rN) with q concurrent processors (r: a constant positive integer). Numerical examples demonstrate that the proposed technique for r = 1 yields significantly faster convergence than not only adaptive PSP with uniform weights, affine projection algorithm, and fast Newton transversal filters but also the regularized recursive least squares algorithm. [ABSTRACT FROM AUTHOR]

This paper studies the application of the linear least-mean-square (LMS) algorithm with leakage to a system having a memoryless saturation-type nonlinearity. This approach represents an interesting alternative to the nonlinear LMS (NLLMS) algorithm. The major drawback to implement the latter is that the model parameters of the system nonlinearity must be known. In contrast, the linear LMS algorithm with leakage does not require such knowledge. It permits to approximate the performance of the NLLMS by properly selecting a constant leakage factor value. To cope with an eventual change of the environment, a strategy for a variable leakage is also proposed. Several numerical simulations are presented with the aim of ratifying the feasibility of the proposed approach. [ABSTRACT FROM AUTHOR]

In some practical applications of array processing, the directions of the incident signals should be estimated adaptively, and/or the time-varying directions should be tracked promptly. In this paper, an adaptive bearing estimation and tracking (ABEST) algorithm is investigated for estimating and tracking the uncorrelated and correlated narrow-band signals impinging on a uniform linear array (ULA) based on the subspace-based method without eigendecomposition (SUMWE), where a linear operator is obtained from the array data to form a basis for the null space by exploiting the array geometry and its shift invariance property. Specifically, the null space is estimated using the least-mean-square (LMS) or normalized LMS (NLMS) algorithm, and the directions are updated using the approximate Newton method. The transient analyses of the LMS and NLMS algorithms are studied, where the "weight" (i.e., the linear operator) is in the form of a matrix and there is a correlation between the "additive noise" and "input data" that involve the instantaneous correlations of the received array data in the updating equation, and the step-size stability conditions are derived explicitly. In addition, the analytical expressions for the mean-square error (MSE) and mean-square deviation (MSD) learning curves of the LMS algorithm are clarified. The effectiveness of the ABEST algorithm is verified, and the theoretical analyses are corroborated through numerical examples. Simulation results show that the ABEST algorithm is computationally simple and has good adaptation and tracking abilities. [ABSTRACT FROM AUTHOR]

In this paper, a new control mechanism for the vari- able forgetting factor (VFF) of the recursive least square (RLS) adaptive algorithm is presented. The control algorithm is basically a gradient-based method of which the gradient is derived from an improved mean square error analysis of RLS. The new mean square error analysis exploits the correlation of the inverse of the correlation matrix with itself that yields improved theoretical results, especially in the transient and steady-state mean square error. It is shown that the theoretical analysis is close to simulation results for different forgetting factors and different model orders. The analysis yields a dynamic equation of mean square error that can be used to derive a dynamic equation of the gradient of mean square error to control the forgetting factor. The dynamic equation can produce a positive gradient when the error is large and a negative gradient when the error is in the steady state. Compared with other variable forgetting factor algorithms, the new control algorithm gives fast tracking and small mean square model error for different signal-to-noise ratios (SNRs). [ABSTRACT FROM AUTHOR]

Subspace tracking is an adaptive signal processing technique useful for a variety of applications. In this paper, we introduce a simple bi-iterative least-square (Bi-LS) method, which is in contrast to the bi-iterative singular value decomposition (Bi-SVD) method. We show that for subspace tracking, the Bi-LS method is easier to simplify than the Bi-SVD method. The linear complexity algorithms based on Bi-LS are computationally more efficient than the existing linear complexity algorithms based on Bi-SVD, although both have the same performance for subspace tracking. A number of other existing subspace tracking algorithms of similar complexity are also compared with the Bi-LS algorithms. [ABSTRACT FROM AUTHOR]

Selection probabilities, viz, rank and sample selection probabilities, quantify the impulse suppression, robustness, low- pass filtering ability, and detail preserving ability of stack filters. Closed-form expressions exist for computing selection probabilities from positive Boolean function describing the stack filter. In this paper, we first present an analysis framework that interprets a Boolean vector to denote membership in a partition set. Using this framework a new algorithm to compute selection probabilities from positive Boolean functions is derived. Next a method to synthesize a stack filter from specified selection. probabilities is presented. An under determined system of linear Diophantine equations is derived in the unknown elements of the C-matrix and the desired selection probabilities. This system of equations is solved to obtain the candidate solutions for the C-matrix. For each candidate C-matrix, an algorithm based on partition sets is used to derive one or more positive Boolean functions of the stack filter or show that none exists. [ABSTRACT FROM AUTHOR]

This paper presents a blind timing phase offset recovery scheme that attempts to optimize the baud spaced equalizer output mean square error (MSE) for a realistic equalizer length that is usually shorter than the ideal length. Among the existing blind timing recovery schemes, few are designed for equalizer output MSE optimization, and none are designed for the realistic case when the equalizer is short. The proposed algorithm (that is based on a cost function that minimizes the dispersion of the received signal) attempts to minimize the MSE of a one-tap equalizer output., It also exhibits good performance for relatively short equalizers. Conditions for the unimodality of the dispersion minimization cost are investigated, and a geometric relationship to the minimum MSE (MMSE) tinting offset is shown qualitatively. The detailed MSE performance of the algorithm is investigated for the representing classes of channels by comparing existing blind timing offset estimation schemes. [ABSTRACT FROM AUTHOR]

Mean squared error (MSE) has been the dominant criterion in adaptive filter theory. A major drawback of the MSE criterion in linear filter adaptation is the parameter bias in the Wiener solution when the input data are contaminated with noise. In this paper, we propose and analyze a new augmented MSE criterion called the Error Whitening Criterion (EWC). EWC is able to eliminate this bias when the noise is white. We will determine the analytical solution of the EWC, discuss some interesting properties, and develop stochastic gradient and other fast algorithms to calculate the EWC solution in an online fashion. The stochastic algorithms are locally computable and have structures and complexities similar to their MSE-based counterparts (LMS and NLMS). Convergence of the stochastic gradient algorithm is established with mild assumptions, and upper bounds on the step sizes are deduced for guaranteed convergence. We will briefly discuss an RLS-like Recursive Error Whitening (REW) algorithm and a minor components analysis (MCA) based EWC-total least squares (TLS) algorithm and further draw parallels between the REW algorithm and the Instrumental Variables (lv) method for system identification. Finally, we will demonstrate the noise-rejection capability of the EWC by comparing the performance with MSE criterion and TLS. [ABSTRACT FROM AUTHOR]

This paper proposes an algebraic solution for the position and velocity of a moving source using the time differences of arrival (TDOAs) and frequency differences of arrival (FDOAs) of a signal received at a number of receivers. The method em- ploys several weighted least-squares minimizations only and does not require initial solution guesses to obtain a location estimate. It does not have the initialization and local convergence problem as in the conventional linear iterative method. The estimated accuracy of the source position and velocity is shown to achieve the Cramér-Rao lower bound for Gaussian TDOA and FDOA noise at moderate noise level before the thresholding effect occurs. Simulations are included to examine the algorithm's performance and compare it with the Taylor-series iterative method. [ABSTRACT FROM AUTHOR]

In this paper, we consider infinite impulse response (IIR) filter design where both magnitude and phase are optimized using a weighted and sampled least-squares criterion. We propose a new convex stability domain defined by positive realness for ensuring the stability of the filter and adapt the Steiglitz-McBride (SM), Gauss-Newton (GN), and classical descent methods to the new stability domain. We show how to describe the stability domain such that the description is suited to semidefinite programming and is implementable exactly; in addition, we prove that this domain contains the domain given by Rouché's theorem. Finally, we give experimental evidence that the best designs are usually obtained with a multistage algorithm, where the three above methods are used in succession, each one being initialized with the result of the previous and where the positive realness stability domain is used instead of that defined by Rouché's theorem. [ABSTRACT FROM AUTHOR]

In this paper, we present mean-squared convergence analysis for the partial-update normalized least-mean square (PU-NLMS) algorithm with closed-form expressions for the case of white input signals. The formulae presented here are more accurate than the ones found in the literature for the PU-NLMS algorithm. Thereafter, the ideas of the partial-update NLMS-type algorithms found in the literature are incorporated in the framework of set-membership filtering, from which data-selective NLMS-type algorithms with partial-update are derived. The new algorithms, referred to herein as the set-membership partial-update normalized least-mean square (SM-PU-NLMS) algorithms, combine the data-selective updating from set-membership filtering with the reduced computational complexity from partial updating. A thorough discussion of the SM-PU-NLMS algorithms follows, whereby we propose different update strategies and provide stability analysis and closed-form formulae for excess mean-squared error (MSE). Simulation results verify the analysis for the PU-NLMS algorithm and the good performance of the SM-PU-NLMS algorithms in terms of convergence speed, final misadjustment, and computational complexity. [ABSTRACT FROM AUTHOR]

The problem of identification/tracking of periodically varying systems is considered. When system coefficients vary rapidly with time, the most frequently used weighted least squares (WLS) and least mean squares (LMS) algorithms are not capable of tracking the changes satisfactorily. To obtain good estimation results, one has to use more specialized adaptive filters, such as the basis function (BF) algorithms, which are based on explicit models of parameter changes. Unfortunately, estimators of this kind are numerically very demanding. The paper describes new recursive algorithms that combine low computational requirements, which are typical of WLS and LMS filters, with very good tracking capabilities, which are typical of BF filters. [ABSTRACT FROM AUTHOR]

We propose an iterative channel estimation algorithm based on the least square estimation (LSE) and sparse message passing (SMP) algorithm for the millimeter wave (mmWave) MIMO systems. The channel coefficients of the mmWave MIMO are approximately modeled as a Bernoulli–Gaussian distribution and the channel matrix is sparse with only a few nonzero entries. By leveraging the advantage of sparseness, we propose an algorithm that iteratively detects the exact locations and values of nonzero entries of the sparse channel matrix. At each iteration, the locations are detected by the SMP, and values are estimated with the LSE. We also analyze the Cramér–Rao Lower Bound (CLRB), and show that the proposed algorithm is a minimum variance unbiased estimator under the assumption that we have the partial priori knowledge of the channel. Furthermore, we employ the Gaussian approximation for message densities under density evolution to simplify the analysis of the algorithm, which provides a simple method to predict the performance of the proposed algorithm. Numerical experiments show that the proposed algorithm has much better performance than the existing sparse estimators, especially when the channel is sparse. In addition, our proposed algorithm converges to the CRLB of the genie-aided estimation of sparse channels with only five turbo iterations. [ABSTRACT FROM AUTHOR]

We show that the generalized total least squares (GTLS) problem with a singular noise covariance matrix is equivalent to the restricted total least squares (RTLS) problem and propose a recursive method for its numerical solution. The method is based on the generalized inverse iteration. The estimation error covariance matrix and the estimated augmented correction are also characterized and computed recursively. The algorithm is cheap to compute and is suitable for online implementation. Simulation results in least squares (LS), data least squares (DLS), total least squares (TLS), and restricted total least squares (RTLS) noise scenarios show fast convergence of the parameter estimates to their optimal values obtained by corresponding batch algorithms. [ABSTRACT FROM PUBLISHER]

Provides information on a study which proposed a joint order detection and blind estimation algorithm for single input multiple output channels. Key notations of the channel model; Geometrical relation between the output and input subspaces; General formulation of least squares smoothing; Comparison of the proposed algorithm with existing techniques; Conclusion.