Search

In this paper, a novel way of deriving proportionate adaptive filters is proposed based on diversity measure minimization using the iterative reweighting techniques well-known in the sparse signal recovery (SSR) area. The resulting least mean square (LMS)-type and normalized LMS (NLMS)-type sparse adaptive filtering algorithms can incorporate various diversity measures that have proved effective in SSR. Furthermore, by setting the regularization coefficient of the diversity measure term to zero in the resulting algorithms, Sparsity promoting LMS (SLMS) and Sparsity promoting NLMS (SNLMS) are introduced, which exploit but do not strictly enforce the sparsity of the system response if it already exists. Moreover, unlike most existing proportionate algorithms that design the step-size control factors based on heuristics, our SSR-based framework leads to designing the factors in a more systematic way. Simulation results are presented to demonstrate the convergence behavior of the derived algorithms for systems with different sparsity levels.

The network of distributed microphone arrays is usually established in an ad hoc manner; hence, network parameters such as the mutual positioning and rotation of the arrays, positions of sources, and synchronization of their recording onset times are initially unknown. In this article, we consider the problem of passively jointly self-calibrating and synchronizing distributed arrays in reverberant rooms. We use a typical two-step approach where, initially, the relative geometry of the network is estimated using Direction of Arrival (DoA) measurements. Subsequently, the absolute scale and synchronization parameters are estimated using Time Difference of Arrival (TDoA) measurements. This article presents methods to improve the robustness and accuracy of estimation of the absolute geometric scaling and synchronization parameters in reverberant conditions, in which TDoA measurements do not follow a normal distribution; furthermore, outliers often occur. To remedy these issues, we propose a Weighted Least Squares (WLS) estimator and schema for weighting the TDoA measurements to increase the estimation accuracy from heteroscedastic TDoA measurements. In addition, we propose an iterative reweighing algorithm with a binary weight to detect and reject TDoA outliers, which exploits the residuals of the parametric model in the least absolute value minimization. A numerical evaluation shows significant improvements in the proposed method over the state of the art in terms of the relative scaling error and mean absolute value of the synchronization parameters.

Periodic detection of wheel tread is necessary for train operation safety. Reference positioning of inner side benchmark is a traditional method for tread detection but there are problems of positioning error caused by field factors in wheel tread dynamic detection such as reference tilt by hunting, foreign matter and light interference. A reference positioning method for wheel tread dynamic detection is proposed in this paper. After extracting the point cloud data through structured light calibration, center-line extraction and other algorithms, reference feature points of the inner side are segmented combined with tread features, and an iterative re-weighted least squares line fitting (IRLS-LF) method is used to realize robust positioning of the inner side benchmark. Under the dynamic tilt condition, the experimental man-machine comparison deviations of flange height and thickness based on IRLS-LF positioning results are ±0.1mm and ±0.2mm respectively, and the both deviation range widths based on LSLF positioning results and fixed-parameter algorithm positioning results are about 0.8 mm. Experimental results show that this method can effectively solve the datum positioning deviation caused by field factors, and effectively ensure the measurement accuracy and robustness of wheel tread dynamic detection.

Sparse hyperspectral unmixing has been a hot topic in recent years. Joint sparsity assumes that each pixel in a small neighborhood of hyperspectral images (HSIs) is composed of the same endmembers, which results in a few nonzero rows in the abundance matrix. Recall that a plethora of unmixing algorithms transform a 3-D HSI into a 2-D matrix with vertical priority. The transformation makes matrix computation easier. It is, however, hard to maintain the horizontal spatial information in HSIs in many cases. To make further use of the spatial information of HSIs, in this article, we propose a bilateral joint-sparse structure for hyperspectral unmixing in an attempt to exploit the local joint sparsity of the abundance matrix in both the vertical and horizontal directions. In particular, we introduce a permutation matrix to realize the bilateral joint-sparse representation and there is no need to construct the matrix explicitly. Moreover, we propose to simultaneously impose the bilateral joint-sparse structure and low rankness on the abundance and develop a new algorithm named bilateral joint-sparse and low-rank unmixing. The proposed algorithm is based on the alternating direction method of multipliers framework and employs a reweighting strategy. The convergence analysis of the proposed algorithm is investigated. Simulated and real-data experiments show the effectiveness of the proposed algorithm.

We propose a new approach to the construction of -means clustering algorithms in which the Mahalanobis distance is used instead of the Euclidean distance. The approach is based on minimizing differentiable estimates of the mean insensitive to outliers. Illustrative examples convincingly show that the proposed algorithm is highly likely to be robust with respect to a large amount of outliers in the data. [ABSTRACT FROM AUTHOR]

In point cloud registration applications, noise and poor initial conditions lead to many false matches. False matches significantly degrade registration accuracy and speed. A penalty function is adopted in many robust point-to-point registration methods to suppress the influence of false matches. However, after applying a penalty function, problems cannot be solved in their analytical forms based on the introduction of nonlinearity. Therefore, most existing methods adopt the descending method. In this paper, a novel iterative-reweighting-based method is proposed to overcome the limitations of existing methods. The proposed method iteratively solves the eigenvectors of a four-dimensional matrix, whereas the calculation of the descending method relies on solving an eight-dimensional matrix. Therefore, the proposed method can achieve increased computational efficiency. The proposed method was validated on simulated noise corruption data, and the results reveal that it obtains higher efficiency and precision than existing methods, particularly under very noisy conditions. Experimental results for the KITTI dataset demonstrate that the proposed method can be used in real-time localization processes with high accuracy and good efficiency. [ABSTRACT FROM AUTHOR]

Optimal designs of finite impulse response (FIR) digital filters with prescribed magnitude and phase responses have found many applications in signal processing. To deal with the nonconvexity of the constraint region in the phase-error and magnitude-error constrained design, a new error constraint method based on a modified annulus sector (MAS) is proposed. With the new constraint method, the phase-error and magnitude-error constrained least squares, phase-error constrained minimax, and iterative reweighted phase-error constrained minimax designs of FIR filters are studied. Analyses and design examples demonstrate the higher accuracy of the MAS constraint and better filter performance by the constraint method than existing constraint methods. An application in the design of digital differentiators with flatness constraints is also provided. [ABSTRACT FROM AUTHOR]

In this paper, we propose a Group-Sparse Representation-based method with applications to Face Recognition (GSR-FR). The novel sparse representation variational model includes a non-convex sparsity-inducing penalty and a robust non-convex loss function. The penalty encourages group sparsity by using an approximation of the $\ell _{0}$ -quasinorm, and the loss function is chosen to make the algorithm robust to noise, occlusions, and disguises. The solution of the non-trivial non-convex optimization problem is efficiently obtained by a majorization-minimization strategy combined with forward-backward splitting, which, in particular, reduces the solution to a sequence of easier convex optimization sub-problems. Extensive experiments on widely used face databases show the potentiality of the proposed model and demonstrate that the GSR-FR algorithm is competitive with the state-of-the-art methods based on sparse representation, especially for very low dimensional feature spaces. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we derive an efficient iterative algorithm for the recovery of block-sparse signals given the finite data alphabet and the non-zero block probability. The non-zero block number is supposed to be far smaller than the total block number (block-sparse). The key principle is the separation of the unknown signal vector into an unknown support vector s and an unknown data symbol vector a . Both number (‖ s ‖ 0) and positions (s i ∈ {0, 1}) of non-zero blocks are unknown. The proposed algorithms use an iterative two-stage LASSO procedure consisting in optimizing the recovery problem alternatively with respect to a and with respect to s . The first algorithm resorts on ℓ 1 -norm of the support vector and the second one applies reweighted ℓ 1 -norm, which further improves the recovery performance. Performance of proposed algorithms is illustrated in the context of sporadic multiuser communications. Simulations show that the reweighted-ℓ 1 algorithm performs close to its lower bound (perfect knowledge of the support vector). [ABSTRACT FROM AUTHOR]

We study sparse solutions of optimal control problems governed by PDEs with uncertain coefficients. We propose two formulations, one where the solution is a deterministic control optimizing the mean objective, and a formulation aiming at stochastic controls that share the same sparsity structure. In both formulations, regions where the controls do not vanish can be interpreted as optimal locations for placing control devices. In this paper, we focus on linear PDEs with linearly entering uncertain parameters. Under these assumptions, the deterministic formulation reduces to a problem with known structure, and thus we mainly focus on the stochastic control formulation. Here, shared sparsity is achieved by incorporating the L1-norm of the mean of the pointwise squared controls in the objective. We reformulate the problem using a norm reweighting function that is defined over physical space only and thus helps to avoid approximation of the random space using samples or quadrature. We show that a fixed point algorithm applied to the norm reweighting formulation leads to a variant of the well-studied iterative reweighted least squares (IRLS) algorithm, and we propose a novel preconditioned Newton-conjugate gradient method to speed up the IRLS algorithm. We combine our algorithms with low-rank operator approximations, for which we provide estimates of the truncation error. We carefully examine the computational complexity of the resulting algorithms. The sparsity structure of the optimal controls and the performance of the solution algorithms are studied numerically using control problems governed by the Laplace and Helmholtz equations. In these experiments the Newton variant clearly outperforms the IRLS method. [ABSTRACT FROM AUTHOR]

This paper addresses the issue of beampattern synthesis with flexible magnitude response and low sidelobe in a frequency diverse array radar. It is formulated as an optimization problem in terms of $\ell _1$-norm minimization, which encourages the sparsity of array pattern. An $\ell _1$ iterative phase compensation (IPC) technique is employed to transform the nonconvex constraint to phase compensation form. Then, to further reduce the sidelobe level, a reweighted $\ell _1$ -IPC algorithm is devised, in which a sparsity-enhanced scheme is utilized to convert the $\ell _1$-norm to a new cost function that more closely resembles $\ell _0$-norm. Numerical results are presented to demonstrate the effectiveness of the proposed approach in different scenarios. [ABSTRACT FROM AUTHOR]

In this paper, the problem of simultaneously estimating the structure and parameters of artificial neural networks with multiple hidden layers is considered. A method based on sparse optimization is proposed. The problem is formulated as an ℓ 0 -norm minimization problem, so that redundant weights are eliminated from the neural network. Such problems are in general combinatorial, and are often considered intractable. Hence, an iterative reweighting heuristic for relaxing the ℓ 0 -norm is presented. Experiments have been carried out on simple benchmark problems, both for classification and regression, and on a case study for estimation of waste heat recovery in ships. All experiments demonstrate the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

Estimating extended brain sources using EEG/MEG source imaging techniques is challenging. EEG and MEG have excellent temporal resolution at millisecond scale but their spatial resolution is limited due to the volume conduction effect. We have exploited sparse signal processing techniques in this study to impose sparsity on the underlying source and its transformation in other domains (mathematical domains, like spatial gradient). Using an iterative reweighting strategy to penalize locations that are less likely to contain any source, it is shown that the proposed iteratively reweighted edge sparsity minimization (IRES) strategy can provide reasonable information regarding the location and extent of the underlying sources. This approach is unique in the sense that it estimates extended sources without the need of subjectively thresholding the solution. The performance of IRES was evaluated in a series of computer simulations. Different parameters such as source location and signal-to-noise ratio were varied and the estimated results were compared to the targets using metrics such as localization error (LE), area under curve (AUC) and overlap between the estimated and simulated sources. It is shown that IRES provides extended solutions which not only localize the source but also provide estimation for the source extent. The performance of IRES was further tested in epileptic patients undergoing intracranial EEG (iEEG) recording for pre-surgical evaluation. IRES was applied to scalp EEGs during interictal spikes, and results were compared with iEEG and surgical resection outcome in the patients. The pilot clinical study results are promising and demonstrate a good concordance between noninvasive IRES source estimation with iEEG and surgical resection outcomes in the same patients. The proposed algorithm, i.e. IRES, estimates extended source solutions from scalp electromagnetic signals which provide relatively accurate information about the location and extent of the underlying source. [ABSTRACT FROM AUTHOR]

Hyperspectral unmixing (HSU) plays an important role in hyperspectral image (HSI) analysis. Recently, the HSU method based on sparse regression has drawn much attention. This paper presents a new weighted sparse regression problem for HSU and proposes two iterative reweighted algorithms for solving this problem, where the weights used for the next iteration are computed from the value of the current solution, and all the mixed pixels of an HSI are unmixed simultaneously. The proposed algorithms can be seen as the combinations of alternating direction method of multipliers and iterative reweighting procedure. Experimental results on both synthetic and real data demonstrate some advantages of the proposed algorithms over some other state-of-the-art sparse unmixing approaches. [ABSTRACT FROM AUTHOR]

In this paper, we propose a reweighted low-rank matrix recovery method and demonstrate its application for robust image restoration. In the literature, principal component pursuit solves low-rank matrix recovery problem via a convex program of mixed nuclear norm and \ell1 norm. Inspired by reweighted \ell1 minimization for sparsity enhancement, we propose reweighting singular values to enhance low rank of a matrix. An efficient iterative reweighting scheme is proposed for enhancing low rank and sparsity simultaneously and the performance of low-rank matrix recovery is prompted greatly. We demonstrate the utility of the proposed method both on numerical simulations and real images/videos restoration, including single image restoration, hyperspectral image restoration, and background modeling from corrupted observations. All of these experiments give empirical evidence on significant improvements of the proposed algorithm over previous work on low-rank matrix recovery. [ABSTRACT FROM PUBLISHER]

Abstract: This paper introduces a novel variational segmentation method within the fuzzy framework, which solves the problem of segmenting multi-region color-scale images of natural scenes. We call this kind of images as natural images. The advantages of our segmentation method are: (1) by introducing the PCA descriptors, our segmentation model can partition color-texture images better than classical variational-based segmentation models, (2) to preserve geometrical structure of each fuzzy membership function, we propose a nonconvex regularization term in our model, (3) to solve our segmentation model more efficiently, we design a fast iteration algorithm in which we integrate the augmented Lagrange multiplier method and the iterative reweighting. We conduct comprehensive experiments to measure the segmentation performance of our model in terms of visual evaluation, and we also demonstrate the efficiency of the corresponding algorithm in terms of a variety of quantitative indices. The proposed model achieves better segmentation results compared with some other well-known models, such as the level-set model and the fuzzy region competition model, while the proposed algorithm is much more efficient than the algorithm of the state-of-the-art natural image segmentation model. [Copyright &y& Elsevier]

The bilateral filter represents a wide group of nonlinear filters for edge-preserving image smoothing. In this work, we study the convergence properties of the bilateral filter algorithm. The understanding is established that the bilateral filter is an optimization procedure. We demonstrate that the bilateral filter is equivalent to minimizing a robust cost criterion using iterative reweighting, which is a good approximation to the very fast but unstable Newton's method. Further, the results of the analysis allow us to derive an improved hybrid smoothing scheme with concerns of computational efficiency and edge preservation. [ABSTRACT FROM AUTHOR]

This paper introduces practical methods of parameter and standard error estimation for adaptive robust regression where errors are assumed to be from a normal/independent family of distributions. In particular, generalized EM algorithms (GEM) are considered for the two cases of t and slash families of distributions. For the t family, one step method is proposed to estimate the degree of freedom parameter. Use of empirical information is suggested for standard error estimation. It is shown that this choice leads to standard errors that can be obtained as a by-product of the GEM algorithm. The proposed methods, as discussed, can be implemented in most available nonlinear regression programs. Details of implementation in SAS NLIN are given using two specific examples. [ABSTRACT FROM AUTHOR]

While deep neural networks (DNNs) have achieved state-of-the-art results in many fields, they are typically over-parameterized. Parameter redundancy, in turn, leads to inefficiency. Sparse signal recovery (SSR) techniques, on the other hand, find compact solutions to overcomplete linear problems. Therefore, a logical step is to draw the connection between SSR and DNNs. In this paper, we explore the application of iterative reweighting methods popular in SSR to learning efficient DNNs. By efficient, we mean sparse networks that require less computation and storage than the original, dense network. We propose a reweighting framework to learn sparse connections within a given architecture without biasing the optimization process, by utilizing the affine scaling transformation strategy. The resulting algorithm, referred to as Sparsity-promoting Stochastic Gradient Descent (SSGD), has simple gradient-based updates which can be easily implemented in existing deep learning libraries. We demonstrate the sparsification ability of SSGD on image classification tasks and show that it outperforms existing methods on the MNIST and CIFAR-10 datasets.

In this letter, we propose a novel conjugate gradient (CG) adaptive filtering algorithm for online estimation of system responses that admit sparsity. Specifically, the Sparsity-promoting Conjugate Gradient (SCG) algorithm is developed based on iterative reweighting methods popular in the sparse signal recovery area. We propose an affine scaling transformation strategy within the reweighting framework, leading to an algorithm that allows the usage of a zero sparsity regularization coefficient. This enables SCG to leverage the sparsity of the system response if it already exists, while not compromising the optimization process. Simulation results show that SCG demonstrates improved convergence and steady-state properties over existing methods.

In this paper, a novel way of deriving proportionate adaptive filters is proposed based on diversity measure minimization using the iterative reweighting techniques well-known in the sparse signal recovery (SSR) area. The resulting least mean square (LMS)-type and normalized LMS (NLMS)-type sparse adaptive filtering algorithms can incorporate various diversity measures that have proved effective in SSR. Furthermore, by setting the regularization coefficient of the diversity measure term to zero in the resulting algorithms, Sparsity promoting LMS (SLMS) and Sparsity promoting NLMS (SNLMS) are introduced, which exploit but do not strictly enforce the sparsity of the system response if it already exists. Moreover, unlike most existing proportionate algorithms that design the step-size control factors based on heuristics, our SSR-based framework leads to designing the factors in a more systematic way. Simulation results are presented to demonstrate the convergence behavior of the derived algorithms for systems with different sparsity levels.

Since computed tomography (CT) was developed over 35 years ago, new mathematical ideas and computational algorithms have been continuingly elaborated to improve the quality of reconstructed images. In recent years, a considerable effort can be noticed to apply the sparse solution of underdetermined system theory to the reconstruction of CT images from undersampled data. Its significance stems from the possibility of obtaining good quality CT images from low dose projections. Among diverse approaches, total variation (TV) minimizing 2D gradients of an image, seems to be the most popular method. In this paper, a new method for CT image reconstruction via sparse gradients estimation (SGE), is proposed. It consists in estimating 1D gradients specified in four directions using the iterative reweighting algorithm. To investigate its properties and to compare it with TV and other related methods, numerical simulations were performed according to the Monte Carlo scheme, using the Shepp-Logan and more realistic brain phantoms scanned at 9-60 directions in the range from 0 to 179°, with measurement data disturbed by additive Gaussians noise characterized by the relative level of 0.1%, 0.2%, 0.5%, 1%, 2% and 5%. The accuracy of image reconstruction was assessed in terms of the relative root-mean-square (RMS) error. The results show that the proposed SGE algorithm has returned more accurate images than TV for the cases fulfilling the sparsity conditions. Particularly, it preserves sharp edges of regions representing different tissues or organs and yields images of much better quality reconstructed from a small number of projections disturbed by relatively low measurement noise., (Copyright © 2016 Elsevier Ltd. All rights reserved.)