Search

In this paper, we study distributed learning with multi-penalty regularization based on a divide-and-conquer approach. Using Neumann expansion and a second order decomposition on difference of operator inverses approach, we derive optimal learning rates for distributed multi-penalty regularization in expectation. As a byproduct, we also deduce optimal learning rates for multi-penalty regularization, which was not given in the literature. These results are applied to the distributed manifold regularization and optimal learning rates are given.

We consider regularization methods based on the coupling of Tikhonov regularization and projection strategies. From the resulting constraint regularization method we obtain level set methods in a straight forward way. Moreover, we show that this approach links the areas of asymptotic regularization to inverse problems theory, scale-space theory to computer vision, level set methods, and shape optimization., Comment: 12 pages, Letter to the editor

In this paper, we consider a time-fractional inverse diffusion problem, where the data are given at x = 1 and the solution is required in the interval 0 x 1 . This problem is typically ill-posed, i.e., the solution (if it exists) does not depend continuously on the data. The simplified Tikhonov regularization method is proposed to solve this problem. An a priori error estimate between the exact solution and its regularized approximation is obtained. Moreover, a new a posteriori parameter choice rule is proposed and the Holder type error estimate is also obtained. Some different type examples are presented to demonstrate the feasibility and efficiency of the proposed method.

Computing steering control for an approximately controllable linear system for a given target state is an ill-posed problem. We use a weighted Tikhonov regularization method and compute the regularized control. It is proved that the target state corresponding to the regularized control is close to the actual state to be attained. We also obtained error estimates and convergence rates involved in the regularization procedure using both the a priori and a posteriori parameter choice rule. Theory is substantiated with numerical experiments.

Tikhonov regularization is one of the most popular methods for solving linear systems of equations or linear least-squares problems with a severely ill-conditioned matrix and an error-contaminated data vector (right-hand side). This regularization method replaces the given problem by a penalized least-squares problem. It is well known that Tikhonov regularization in standard form may yield approximate solutions that are too smooth, i.e., the computed approximate solution may lack many details that the desired solution of the associated, but unavailable, error-free problem might possess. Fractional Tikhonov regularization methods have been introduced to remedy this shortcoming. However, the computed solution determined by fractional Tikhonov methods in standard form may display undesirable spurious oscillations. This paper proposes that fractional Tikhonov methods be equipped with a nonlinear penalty term, such as a TV-norm penalty term, to reduce unwanted oscillations. Numerical examples illustrate the benefits of this approach.

We consider the solution of large linear systems of equations that arise from the discretization of ill-posed problems. The matrix has a Kronecker product structure and the right-hand side is contaminated by measurement error. Problems of this kind arise, for instance, from the discretization of Fredholm integral equations of the first kind in two space-dimensions with a separable kernel and in image restoration problems. Regularization methods, such as Tikhonov regularization, have to be employed to reduce the propagation of the error in the right-hand side into the computed solution. We investigate the use of the global GolubKahan bidiagonalization method to reduce the given large problem to a small one. The small problem is solved by employing Tikhonov regularization. A regularization parameter determines the amount of regularization. The connection between global GolubKahan bidiagonalization and Gauss-type quadrature rules is exploited to inexpensively compute bounds that are useful for determining the regularization parameter by the discrepancy principle.

Computation of control for a controlled partial differential equation is a difficult task, especially when the control problem is ill posed. In this paper, we propose a method of computing the regula...

In this study Tikhonov Gradient type-method is considered for nonlinear ill-posed operator equations. In our convergence analysis, we use hypotheses only on the first Frechet derivative of F in contrast to the higher order Frechet derivatives used in the earlier studies. We obtained ‘optimal’ order error estimate by choosing the regularization parameter according to the adaptive method proposed by Pereverzev and Schock (SIAM J Numer Anal 43(5):2060–2076, 2005).

In this study, we apply Tikhonov’s regularization algorithm for a 3-D density inversion from the gravity-gradiometry data. To reduce the non-uniqueness of the inverse solution (carried out without additional information from geological evidence), we implement the depth-weighting empirical function. However, the application of an empirical function in the inversion equation brings the bias problem of the regularization factor when a traditional Tikhonov’s algorithm is applied. To solve the bias problem of regularization factor selection, we present a standardized solution that comprises two parts for solving a 3-D constrained inversion equation, specifically the linear matrix transformation and Tikhonov’s regularization algorithm. Since traditional regularization techniques become numerically inefficient when dealing with large number of data, we further apply methods which include the Simultaneous Iterative Reconstruction Technique (SIRT) and the wavelet compression combined with Least Squares QR-decomposition (LSQR). In our simulation study, we demonstrate that SIRT as well as the wavelet compression plus LSQR algorithm improve the computation efficiency, while provide results which closely agree with that obtained from applying Tikhonov’s regularization. In particular, the algorithm of wavelet compression plus LSQR shows the best computing efficiency, because it combines the advantages of coefficients compression of big matrix and fast solution of sparse matrix. Similar findings are confirmed from the vertical gravity gradient data inversion for detecting potential deposits at the Kauring (near Perth, Western Australia) testing site.

Ill-posed problems are widely existed in signal processing. In this paper, we review popular regularization models such as truncated singular value decomposition regularization, iterative regularization, variational regularization. Meanwhile, we also retrospect popular optimization approaches and regularization parameter choice methods. In fact, the regularization problem is inherently a multi-objective problem. The traditional methods usually combine the fidelity term and the regularization term into a single-objective with regularization parameters, which are difficult to tune. Therefore, we propose a multi-objective framework for ill-posed problems, which can handle complex features of problem such as non-convexity, discontinuity. In this framework, the fidelity term and regularization term are optimized simultaneously to gain more insights into the ill-posed problems. A case study on signal recovery shows the effectiveness of the multi-objective framework for ill-posed problems.

Tikhonov regularization is commonly used for the solution of linear discrete ill-posed problems with error-contaminated data. A regularization parameter that determines the quality of the computed solution has to be chosen. One of the most popular approaches to choosing this parameter is to minimize the Generalized Cross Validation (GCV) function. The minimum can be determined quite inexpensively when the matrix A that defines the linear discrete ill-posed problem is small enough to rapidly compute its singular value decomposition (SVD). We are interested in the solution of linear discrete ill-posed problems with a matrix A that is too large to make the computation of its complete SVD feasible, and show how upper and lower bounds for the numerator and denominator of the GCV function can be determined fairly inexpensively for large matrices A by computing only a few of the largest singular values and associated singular vectors of A. These bounds are used to determine a suitable value of the regularization parameter. Computed examples illustrate the performance of the proposed method.

Inverse problems of identifying parameters in partial differential equations constitute an important class of problems with diverse real-world applications. These identification problems are commonly explored in an optimization framework and there are many optimization formulations having their own advantages and disadvantages. Although a non-convex output least-squares (OLS) objective is commonly used, a convex-modified output least-squares (MOLS) has shown encouraging results in recent years. In this work, we focus on various aspects of the MOLS approach. We devise a rigorous (quadratic and non-quadratic) regularization framework for the identification of smooth as well as discontinuous coefficients. This framework subsumes the total variation regularization that has attracted a great deal of attention in identifying sharply varying coefficients and also in image processing. We give new existence results for the regularized optimization problems for OLS and MOLS. Restricting to the Tikhonov (qua...

The well-known approach to solve the ill-posed problem is Tikhonov regularization scheme. But, the approximate solution of Tikhonov scheme may not contain all the details of the exact solution. To circumference this problem, weighted Tikhonov regularization has been introduced. In this article, we propose two a posteriori parameter choice rules to choose the regularization parameter for weighted Tikhonov regularization and establish the optimal rate of convergence $$O(\delta ^\frac{\alpha +1}{\alpha +2})$$ for the scheme based on these proposed rules. The numerical results are documented to demonstrate the significance of the theoretical results.

In this paper, a numerical method is proposed for the numerical solution of a linear system with suitable initial and boundary conditions using the cubic B-spline collocation scheme to determine th...

Numerical reconstruction of the missing boundary data in 2D and 3D linear elasticity.Noisy over-prescribed data are available on the remaining accessible boundary.The inverse problem is solved by a meshless method, i.e. the method of fundamental solutions (MFS).Stabilization is achieved via several SVD-based regularization methods.The regularization parameter is selected via the criteria used for inverse problems. The numerical reconstruction of the missing Dirichlet and Neumann data on an inaccessible part of the boundary in the case of two- and three-dimensional linear isotropic elastic materials from the knowledge of over-prescribed noisy measurements taken on the remaining accessible boundary part is investigated. This inverse problem is solved using the method of fundamental solutions (MFS), whilst its stabilization is achieved through several singular value decomposition (SVD)-based regularization methods, such as the Tikhonov regularization methodź48, the damped SVD and the truncated SVDź18. The regularization parameter is selected according to the discrepancy principleź40, generalized cross-validation criterionź14 and Hansen's L-curve methodź20.

Both structured componentwise and structured normwise perturbation analysis of the Tikhonov regularization are presented. The structured matrices under consideration include: Toeplitz, Hankel, Vandermonde, and Cauchy matrices. Structured normwise, mixed and componentwise condition numbers for the Tikhonov regularization are introduced and their explicit expressions are derived. For the general linear structure, based on the derived expressions, we prove structured condition numbers are smaller than their corresponding unstructured counterparts. By means of the power method and small sample statistical condition estimation (SCE), fast condition estimation algorithms are proposed. Our estimation methods can be integrated into Tikhonov regularization algorithms that use the generalized singular value decomposition (GSVD). For large scale linear structured Tikhonov regularization problems, we show how to incorporate the SCE into the preconditioned conjugate gradient (PCG) method to get the posterior error estimations. The structured condition numbers and perturbation bounds are tested on some numerical examples and compared with their unstructured counterparts. Our numerical examples demonstrate that the structured mixed condition numbers give sharper perturbation bounds than existing ones, and the proposed condition estimation algorithms are reliable. Also, an image restoration example is tested to show the effectiveness of the SCE for large scale linear structured Tikhonov regularization problems.

First-arrival traveltime tomography is a robust tool for near-surface velocity estimation. A common approach to stabilizing the ill-posed inverse problem is to apply Tikhonov regularization to the inversion. However, the Tikhonov regularization method recovers smooth local structures while blurring the sharp features in the model solution. We present a first-arrival traveltime tomography method with modified total-variation regularization to preserve sharp velocity contrasts and improve the accuracy of velocity inversion. To solve the minimization problem of the new traveltime tomography method, we decouple the original optimization problem into the two following subproblems: a standard traveltime tomography problem with the traditional Tikhonov regularization and a L2 total-variation problem. We apply the conjugate gradient method and split-Bregman iterative method to solve these two subproblems, respectively. Our synthetic examples show that the new method produces higher resolution models than the conventional traveltime tomography with Tikhonov regularization, and creates less artefacts than the total variation regularization method for the models with sharp interfaces. For the field data, pre-stack time migration sections show that the modified total-variation traveltime tomography produces a near-surface velocity model, which makes statics corrections more accurate.

In this paper, we study a fractional Tikhonov regularization method (FTRM) for solving a Cauchy problem of Helmholtz equation in the frequency domain. On the one hand, the FTRM retains the advantage of classical Tikhonov method. On the other hand, our method can prevent the effect of oversmoothing of classical Tikhonov method and conveniently control the amount of damping. The convergence error estimates between the exact solution and its regularization approximation are constructed. Several interesting numerical examples are provided, which validate the effectiveness of the proposed method.

In this paper, we study a non-characteristic Cauchy problem for a radially symmetric inverse heat conduction equation in a two-layer domain. This is a severely ill-posed problem in the sense that the solution (if it exists) does not depend continuously on the data. It is well-known that the classical Tikhonov regularization solutions are too smooth and the approximate solutions may lack details that might be contained in the exact solutions. Combining Fourier transform technique with a modified version of the classical Tikhonov regularization, we obtain a regularized solution which is stably convergent to the exact solution with a sharp error estimate.

In this paper we establish the error estimates for multi-penalty regularization under the general smoothness assumption in the context of learning theory. One of the motivation for this work is to study the convergence analysis of two-parameter regularization theoretically in the manifold learning setting. In this spirit, we obtain the error bounds for the manifold learning problem using more general framework of multi-penalty regularization. We propose a new parameter choice rule "the balanced-discrepancy principle" and analyze the convergence of the scheme with the help of estimated error bounds. We show that multi-penalty regularization with the proposed parameter choice exhibits the convergence rates similar to single-penalty regularization. Finally on a series of test samples we demonstrate the superiority of multi-parameter regularization over single-penalty regularization.

In this paper we use the L-curve method and the Morozov discrepancy principle for the estimation of the regularization parameter in the regularization of time-delayed optimal control computation. Zeroth order, first order and second order differential operators are considered. Two test examples show that the L-curve method and the two discrepancy principles give close estimations for the regularization parameters.

In this paper, we study an inverse problem of identifying a time-dependent term of an unknown source for a time fractional diffusion equation using nonlocal measurement data. Firstly, we establish the conditional stability for this inverse problem. Then two regularization methods are proposed to for reconstructing the time-dependent source term from noisy measurements. The first method is an integral equation method which formulates the inverse source problem into an integral equation of the second kind; and a prior convergence rate of regularized solutions is derived with a suitable choice strategy of regularization parameters. The second method is a standard Tikhonov regularization method and formulates the inverse source problem as a minimizing problem of the Tikhonov functional. Based on the superposition principle and the technique of finite-element interpolation, a numerical scheme is proposed to implement the second regularization method. One- and two-dimensional examples are carried out to verify ...

Electrical resistance tomography (ERT) reconstructs the conductivity distribution from the boundary changes of electrical measurements. The inverse problem of ERT is seriously ill-posed where regularization methods are needed to treat this ill-posedness. A proper choice of regularization parameter which controls the degree of smoothing is very important for these regularization methods. Although have been a variety of methods, such as L-curve method, to choose a reasonable parameter for the problem, these methods usually result in a scalar parameter which cannot distinctly express the spatial characteristic of the conductivity distribution. So a spatially adaptive regularization parameter choice method is proposed for regularizing the inverse problem of ERT based on Tikhonov regularization. Since large regularization parameters can stabilize and smoothen the solution, while small regularization parameters can approximate and sharpen the solution, the proposed method adaptively updates the regularization parameters during the iteration process and provides spatially varying parameter for each pixel of the reconstructed image. When the iteration is stopped, large regularization parameters for the smooth background region and small regularization parameters for the object region can be obtained. The method is discussed using simulated data for some typical conductivity distributions, and further applied to the analysis of real measurement data acquiring from the practical system. The results demonstrate that flexible regularization parameter vectors can be achieved for different distributions and the strength of regularization is adaptively provided for different regions in a specific distribution. The adaptive method achieves an efficient and reliable regularization solution and has outstanding performance in noise immunity especially in smooth background regions.

The numerical solution of linear discrete ill-posed problems typically requires regularization, i.e., replacement of the available ill-conditioned problem by a nearby better conditioned one. The most popular regularization methods for problems of small to moderate size are Tikhonov regularization and truncated singular value decomposition (TSVD). By considering matrix nearness problems related to Tikhonov regularization, several novel regularization methods are derived. These methods share properties with both Tikhonov regularization and TSVD, and can give approximate solutions of higher quality than either one of these methods.

In this paper, we consider a Cauchy problem for the semi-linear elliptic equation and use a modified boundary Tikhonov-type regularization method to overcome its ill-posedness. The existence, uniqueness and stability of the regularization solution are proven. Under an a-priori bound assumption for the exact solution, a convergence estimate of Holder type for this method is obtained. Finally, an iterative scheme is proposed to calculate the regularization solution, numerical results show that this method works well.

The set of linear equations in the inversion of particle size distribution (PSD) based on forward light scattering is an ill-posed problem. In order to solve the inverse problem of this kind, a number of inversion algorithms have been proposed. The regularization algorithm can reconstruct the PSD, but in usual case, the solution may contain negative values and is strongly oscillating. Owing to the natural reason, the solution should be non-negative and smooth. In this paper, a simple non-negative constraint (NNC) is used with a combination of the Tikhonov regularization. Simulations and experiments show that the regularization with NNC can achieve more reasonable results.

In this paper we consider projection techniques to obtain the finite dimensional realization of a Tikhonov gradient type-method considered in George et al. (Local convergence of a Tikhonov gradient type-method under weak conditions, communicated, 2016) for approximating a solution $$\hat{x}$$ of the nonlinear ill-posed operator equation $$F(x)=y$$ . The main advantage of the proposed method is that the inverse of the operator F is not involved in the method. The regularization parameter is chosen according to the adaptive method considered by Pereverzev and Schock (SIAM J Numer Anal 43(5):2060–2076, 2005). We also derive optimal stopping conditions on the number of iterations necessary for obtaining the optimal order of convergence. Using two numerical examples we compare our results with an existing method to justify the theoretical results.

We investigate the optimum correction of an absolute value equation by minimally changing the coefficient matrix and right-hand side vector using Tikhonov regularization. Solving this problem is equivalent to minimizing the sum of fractional quadratic and quadratic functions. The primary difficulty with this problem is its nonconvexity. Nonetheless, we show that a global optimal solution to this problem can be found by solving an equation on a closed interval using the subgradient method. Some examples are provided to illustrate the efficiency and validity of the proposed method.

Purpose – The linear regression technique is widely used to determine empirical parameters of fatigue life profile while the results may not continuously depend on experimental data. Thus Tikhonov-Morozov method is utilized here to regularize the linear regression results and consequently reduces the influence of measurement noise without notably distorting the fatigue life distribution. The paper aims to discuss these issues. Design/methodology/approach – Tikhonov-Morozov regularization method would be shown to effectively reduce the influences of measurement noise without distorting the fatigue life distribution. Moreover since iterative regularization methods are known to be an attractive alternative to Tikhonov regularization, four gradient iterative methods called as simple iteration, minimum error, steepest descent and conjugate gradient methods are examined with an appropriate initial guess of regularized coefficients. Findings – It has been shown that in case of sparse fatigue life measurements, linear regression results may not have continuous dependence on experimental data and measurement error could lead to misinterpretations of the solution. Therefore from engineering safety point of view, utilizing regularization method could successfully reduce the influence of measurement noise without significantly distorting the fatigue life distribution. Originality/value – An excellent initial guess for mixed iterative-direct algorithm is introduced and it has been shown that the combination of Newton iterative approach and Morozov discrepancy principle is one of the interesting strategies for determination of regularization parameter having an excellent rate of convergence. Moreover since iterative methods are known to be an attractive alternative to Tikhonov regularization, four gradient descend methods are examined here for regularization of the linear regression problem. It has been found that all of gradient decent methods with an appropriate initial guess of regularized coefficients have an excellent convergence to Tikhonov-Morozov regularization results.

We address the classical issue of appropriate choice of the regularization and discretization level for the Tikhonov regularization of an inverse problem with imperfectly measured data. We focus on the fact that the proper choice of the discretization level in the domain together with the regularization parameter is a key feature in adequate regularization. We propose a discrepancy-based choice for these quantities by applying a relaxed version of Morozov's discrepancy principle. Indeed, we prove the existence of the discretization level and the regularization parameter satisfying such discrepancy. We also prove associated regularizing properties concerning the Tikhonov minimizers., Comment: 28 pages, 4 figures

The article deals with one of inverse problems of elastography: knowing displacement of compressed tissue finds the distribution of Young’s modulus in the investigated specimen. The direct problem is approximated and solved by the finite element method. The inverse problem can be stated in different ways depending on whether the solution to be found is smooth or discontinuous. Tikhonov regularization with appropriate regularizing functionals is applied to solve these problems. In particular, discontinuous Young’s modulus distribution can be found on the class of 2D functions with bounded variation of Hardy–Krause type. It is shown in the paper that a variant of Tikhonov regularization provides for such discontinuous distributions the so-called piecewise uniform convergence of approximate solutions as the error levels of the data vanish. The problem of practical a posteriori estimation of the accuracy for obtained approximate solutions is under consideration as well. A method of such estimation is presente...

In this paper, we consider an unknown source problem for the modified Helmholtz equation. The Tikhonov regularization method in Hilbert scales is extended to deal with ill-posedness of the problem. An a priori strategy and an a posteriori choice rule have been present to obtain the regularization parameter and corresponding error estimates have been obtained. The smoothness parameter and the a priori bound of exact solution are not needed for the a posteriori choice rule. Numerical results are presented to show the stability and effectiveness of the method.

Based on kernel-based approximation technique, we devise in this paper an efficient and accurate numerical scheme for solving a backward space-time fractional diffusion problem (BSTFDP). The kernels used in the approximation are the fundamental solutions of the space-time fractional diffusion equation expressed in terms of inverse Fourier transform of Mittag-Leffler functions. The use of Inverse fast Fourier transform (IFFT) technique enables an accurate and efficient evaluation of the fundamental solutions and gives a robust numerical algorithm for the solution of the BSTFDP. Since the BSTFDP is intrinsic ill-posed, we apply the standard Tikhonov regularization technique to obtain a stable solution to the highly ill-conditioned resultant system of linear equations. For choosing optimal regularization parameter, we combine the regularization technique with the generalized cross validation (GCV) method for an optimal placement of the source points in the use of fundamental solutions. Meanwhile, the proposed algorithm also speeds up the previous method given in Dou and Hon (2014). Several numerical examples are constructed to verify the accuracy and efficiency of the proposed method.

A Tikhonov finite-dimensional approximation is applied to a Fredholm integral equation of the first kind. This allows using a variational regularization method with a regularization parameter from the residual principle and reducing the problem to a system of linear algebraic equations. The accuracy of the approximate solution is estimated with allowance for the error of the finitedimensional approximation of the problem. The use of this approach is illustrated by solving an inverse boundary value problem for the heat conduction equation.

This paper proposes a new approach for choosing the regularization parameters in multi-parameter regularization methods when applied to approximate the solution of linear discrete ill-posed problems. We consider both direct methods, such as Tikhonov regularization with two or more regularization terms, and iterative methods based on the projection of a Tikhonov-regularized problem onto Krylov subspaces of increasing dimension. The latter methods regularize by choosing appropriate regularization terms and the dimension of the Krylov subspace. Our investigation focuses on selecting a proper set of regularization parameters that satisfies the discrepancy principle and maximizes a suitable quantity, whose size reflects the quality of the computed approximate solution. Theoretical results are shown and illustrated by numerical experiments.

Electrical resistance tomography (ERT) is a promising measurement technique in industrial process imaging. However, image reconstruction in ERT is an ill-posed inverse problem. Regularization methods have been developed to solve the ill-posed inverse problem. Since the penalty term is a form of L2-norm, Tikhonov regularization method guarantees the stability of the solution, but it always makes the image edge oversmoothed. Total variation (TV) regularization method has good ability of preserving image edges. A hybrid regularization method, which combines Tikhonov with TV regularization method, is proposed to get better reconstructed images. The choice of the adaptive weighted parameter between TV and Tikhonov penalty term has been discussed in detail. In the proposed hybrid regularization method, the function of conductivity gradients is used as the adaptive weighted parameter to control automatically the weighting between the penalty terms from TV and Tikhonov regularization. For the model with sharp edges, the proportion of the penalty term from TV regularization is increased to preserve the edges, while for the model with smooth edges, the proportion of penalty term from Tikhonov regularization is increased to make the solution stable and robust to noise. Both simulation and experimental results of Tikhonov, TV and hybrid regularization method are shown respectively, which indicates that the hybrid regularization method can improve the reconstruction quality with sharp edges and is more robust to noise, and it is applicable for models with different edge characteristic.

This paper discusses the properties of certain risk estimators that recently regained popularity for choosing regularization parameters in ill-posed problems, in particular for sparsity regularization. They apply Stein’s unbiased risk estimator (SURE) to estimate the risk in either the space of the unknown variables or in the data space. We will call the latter PSURE in order to distinguish the two different risk functions. It seems intuitive that SURE is more appropriate for ill-posed problems, since the properties in the data space do not tell much about the quality of the reconstruction. We provide theoretical studies of both approaches for linear Tikhonov regularization in a finite dimensional setting and estimate the quality of the risk estimators, which also leads to asymptotic convergence results as the dimension of the problem tends to infinity. Unlike previous works which studied single realizations of image processing problems with a very low degree of ill-posedness, we are interested in the statistical behaviour of the risk estimators for increasing ill-posedness. Interestingly, our theoretical results indicate that the quality of the SURE risk can deteriorate asymptotically for ill-posed problems, which is confirmed by an extensive numerical study. The latter shows that in many cases the SURE estimator leads to extremely small regularization parameters, which obviously cannot stabilize the reconstruction. Similar but less severe issues with respect to robustness also appear for the PSURE estimator, which in comparison to the rather conservative discrepancy principle leads to the conclusion that regularization parameter choice based on unbiased risk estimation is not a reliable procedure for ill-posed problems. A similar numerical study for sparsity regularization demonstrates that the same issue appears in non-linear variational regularization approaches.

This work is concerned with linear inverse problems where a distributed parameter is known a priori to only take on values from a given discrete set. This property can be promoted in Tikhonov regularization with the aid of a suitable convex but nondifferentiable regularization term. This allows applying standard approaches to show well-posedness and convergence rates in Bregman distance. Using the specific properties of the regularization term, it can be shown that convergence (albeit without rates) actually holds pointwise. Furthermore, the resulting Tikhonov functional can be minimized efficiently using a semi-smooth Newton method. Numerical examples illustrate the properties of the regularization term and the numerical solution.

Tikhonov regularization is a popular method for the solution of linear discrete ill-posed problems with error-contaminated data. Nonstationary iterated Tikhonov regularization is known to be able to determine approximate solutions of higher quality than standard Tikhonov regularization. We investigate the choice of solution subspace in iterative methods for nonstationary iterated Tikhonov regularization of large-scale problems. Generalized Krylov subspaces are compared with Krylov subspaces that are generated by Golub---Kahan bidiagonalization and the Arnoldi process. Numerical examples illustrate the effectiveness of the methods.

Based on the smoothing Newton method and the Tikhonov regularization method, we construct a regularization Newton method for the second-order cone complementarity problem. The method uses a non-monotone line search scheme which contains the usual monotone line search as a special case. By using the theory of Euclidean Jordan algebras, we prove that the proposed method is globally and locally quadratically convergent under suitable assumptions. Some numerical results are reported which indicate the effectiveness of the method.

In this paper we consider the training of single hidden layer neural networks by pseudoinversion, which, in spite of its popularity, is sometimes affected by numerical instability issues. Regularization is known to be effective in such cases, so that we introduce, in the framework of Tikhonov regularization, a matricial reformulation of the problem which allows us to use the condition number as a diagnostic tool for identification of instability. By imposing well-conditioning requirements on the relevant matrices, our theoretical analysis allows the identification of an optimal value for the regularization parameter from the standpoint of stability. We compare with the value derived by cross-validation for overfitting control and optimisation of the generalization performance. We test our method for both regression and classification tasks. The proposed method is quite effective in terms of predictivity, often with some improvement on performance with respect to the reference cases considered. This approach, due to analytical determination of the regularization parameter, dramatically reduces the computational load required by many other techniques., Published on Neural Networks

This paper is concerned with the image deconvolution problem. For the basic model, where the convolution matrix can be diagonalized by discrete Fourier transform, the Tikhonov regularization method is computationally attractive since the associated linear system can be easily solved by fast Fourier transforms. On the other hand, the provided solutions are usually oversmoothed and other regularization terms are often employed to improve the quality of the restoration. Of course, this weighs down on the computational cost of the regularization method. Starting from the fact that images have sparse representations in the Fourier and wavelet domains, many deconvolution methods have been recently proposed with the aim of minimizing the l1-norm of these transformed coefficients. This paper uses the iteratively reweighted least squares strategy to introduce a diagonal weighting matrix in the Fourier domain. The resulting linear system is diagonal and hence the regularization parameter can be easily estimated, for instance by the generalized cross validation. The method benefits from a proper initial approximation that can be the observed image or the Tikhonov approximation. Therefore, embedding this method in an outer iteration may yield further improvement of the solution. Finally, since some properties of the observed image, like continuity or sparsity, are obviously changed when working in the Fourier domain, we introduce a filtering factor which keeps unchanged the large singular values and preserves the jumps in the Fourier coefficients related to the low frequencies. Numerical examples are given in order to show the effectiveness of the proposed method.

International audience; This paper presents an alternative approach to the regularized least squares solution of ill-posed inverse problems. Instead of solving a minimization problem with an objective function composed of a data term and a regularization term, the regularization information is used to define a projection onto a convex subspace of regularized candidate solutions. The objective function is modified to include the projection of each iterate in the place of the regularization. Numerical experiments based on the problem of motion estimation for geophysical fluid images, show the improvement of the proposed method compared with regularization methods. For the presented test case, the incremental projection method uses 7 times less computation time than the regularization method, to reach the same error target. Moreover, at convergence, the incremental projection is two order of magnitude more accurate than the regularization method.

Summary Structural damage identification is essentially an inverse problem. Ill-posedness is a common obstacle encountered in solving such an inverse problem, especially in the context of a sensitivity-based model updating for damage identification. Tikhonov regularization, also termed as l2-norm regularization, is a common approach to handle the ill-posedness problem and yields an acceptable and smooth solution. Tikhonov regularization enjoys a more popular application as its explicit solution, computational efficiency, and convenience for implementation. However, as the l2-norm term promotes smoothness, the solution is sometimes over smoothed, especially in the case that the sensor number is limited. On the other side, the solution of the inverse problem bears sparse properties because typically, only a small number of components of the structure are damaged in comparison with the whole structure. In this regard, this paper proposes an alternative way, sparse regularization, or specifically l1-norm regularization, to handle the ill-posedness problem in response sensitivity-based damage identification. The motivation and implementation of sparse regularization are firstly introduced, and the differences with Tikhonov regularization are highlighted. Reweighting sparse regularization is adopted to enhance the sparsity in the solution. Simulation studies on a planar frame and a simply supported overhanging beam show that the sparse regularization exhibits certain superiority over Tikhonov regularization as less false-positive errors exist in damage identification results. The experimental result of the overhanging beam further demonstrates the effectiveness and superiorities of the sparse regularization in response sensitivity-based damage identification. Copyright © 2015 John Wiley & Sons, Ltd.

The severe effects of several types of noise present in tomographic image reconstruction are well known. Some of these are a consequence of the ill-posedness of this inverse problem. In this work, we investigate the impact of a Tikhonov regularization on the solution of a gamma-ray tomography reconstruction by means of a least squares numerical method. The theoretical methodology is considered in a broad sense as a Tikhonov regularization, but also includes the Morozov concept used specifically for the delta parameter control. The reconstruction quality shows effective improvement when this technique is applied to simple gamma-ray tomography algorithms. Furthermore, the impact of these regularization techniques on the solutions of linear systems of equations is significant. An ART (Algebraic Reconstruction Technique)-type algorithm was used for the reconstruction of simulated data utilizing built-in Matlab functions. These were compared with data obtained through a regularization implemented with TSVD (Truncated Singular Value Decomposition), as well as data obtained through hybrid algorithms such as TSVD plus Toepelitz, tridiagonal and identity operators. The quality of the resulting reconstruction is evaluated through RMSE (Root Mean Square Error). Direct comparisons suggest that for a high noise level and high delta parameter the TSVD plus tridiagonal operator is the best choice.

This paper presents a nonstationary iterated Tikhonov regularization method for the solution of large-scale Tikhonov minimization problems in general form. The method projects the large-scale problem into a sequence of generalized Krylov subspaces of low dimension. The regularization parameter is determined by the discrepancy principle. Numerical examples illustrate the effectiveness of the method.

The determination of space-dependent functions from boundary measurements or inner pointwise measurements is ill-posed inverse problems that require regularization tools to be stabilized. Among numerous regularization strategies, the Tikhonov penalization is one of the most used in the field of space-dependent function estimation. Its efficient use relies on the Tikhonov parameter value for which search is time consuming although necessary, especially in the field of nonlinear inversion. Other strategies, such as appropriate parameterization, have recently proven to be very efficient to cope with the ill-posedness of such problems. This paper shows that the optimal Tikhonov parameter is almost independent of the mesh used to project the functions to be retrieved. As a consequence, this value should be seeked using a coarse mesh even though reconstructions could further be done on finer meshes. This conclusion is validated by numerical means.

In this paper, we consider a backward problem for a time-fractional diffusion equation. Such a problem is ill-posed. The optimal error bound for the problem under a source condition is analyzed. A simplified Tikhonov regularization method is utilized to solve the problem, and its convergence rates are analyzed under an a priori regularization parameter choice rule and an a posteriori regularization parameter choice rule, respectively. Numerical examples show that the proposed regularization method is effective and stable, and both parameter choice rules work well.

Numerical problems, including linear algebraic equations, are frequently ill-posed, which means that small perturbations in the observation data can have large effects on the computed solutions. In these cases some kind of regularization method must be used to find a feasible solution. Regularization implies the modification of the original ill-posed problem, so that it becomes well-posed, and the Tikhonov regularization method is the most often used. This method requires the choice of a regularization parameter, and this choice is a crucial issue. Recently, several papers were presented proposing adaptive Tikhonov regularization parameters, instead of constant ones. This paper revisits all these adaptive choices and shows that they can be interpreted in a unified and systematic manner using the Liapunov Optimizing Control (LOC) method, which, in addition, opens up the possibility of new choices of the regularization parameter. Examples of matrices from real applications show that one of the methods designed by the LOC method, called the OVM method by its discoverer, can even outperform the CG method for some matrices, although the CG method performs better for a larger class of matrices.

In this paper, in order to solve mathematical programs with generalized nonlinear comple-mentarity constraints, we present a new regularization method of Tikhonov type, based on regularizing an optimization problem with equality constraints. The stability and convergence of the regularized solutions are considered. Numerical examples are given for illustration.