Search

AbstractIn contrast to using means, there are two distinct approaches to developing robust measures effect size when dealing with two dependent variables. One is based on difference scores and the other focuses on the marginal distributions. The former approach is trivial based on extant results. This paper focuses on new robust methods based on the marginal distributions that take into account the strength of the association between the two dependent variables. Simulations are used to assess the extent a reasonably accurate confidence interval can be computed. Included is a proposed method for situations where there is a covariate. [ABSTRACT FROM AUTHOR]

We consider the multiple change point problem in a general framework based on estimating equations. This extends classical sample mean-based methodology to include robust methods but also different types of changes such as changes in linear regression or changes in count data including Poisson autoregressive time series. In this framework, we derive a general theory proving consistency for the number of change points and rates of convergence for the estimators of the locations of the change points. More precisely, two different types of MOSUM (moving sum) statistics are considered: A MOSUM-Wald statistic based on differences of local estimators and a MOSUM-score statistic based on a global inspection parameter. The latter is usually computationally less involved in particular in nonlinear problems where no closed form of the estimator is known such that numerical methods are required. Finally, we evaluate the methodology by some simulations as well as using geophysical well-log data. [ABSTRACT FROM AUTHOR]

The work develops the theory of stable M-estimators belonging to the class of redescending estimators, having the property of resistance to asymmetric contamination. Many well-known redescending estimators can be obtained within the framework of the locally stable approach of A.M. Shurygin, based on the analysis of the estimator instability functional (L2-norm of the influence function), or his approach based on the model of a series of samples with random point contamination (point Bayesian contamination model). These approaches are convenient for constructing various stable M-estimators and, in comparison with classical robust procedures, provide wider opportunities. The family of conditionally optimal estimators proposed by A.M. Shurygin within the framework of the first of the listed approaches can be defined as optimizing the asymptotic dispersion under a constraint on the value of instability. The corresponding problem can be represented in the form of optimization of the weighted L2-norm of the influence function. The second approach considers a specially formed nonparametric neighborhood of the model distribution, and it can also be reduced to the analysis of the weighted L2-norm of the influence function. Thus, this estimation quality criterion is quite general and useful for constructing robust estimators. The theory of estimators that are optimal in terms of weighted L2-norm of the influence function is currently underdeveloped. Specifically, for the corresponding families of estimators, the question of the uniqueness of family members remains unresolved. The question comes down to studying the convexity (concavity) of the optimized functional depending on the parameter defining the family. In the presented work, an expression is obtained in general form for the derivative with respect to the parameter of the quality functional of the optimal estimator. Inequalities are obtained for the second derivative necessary to establish its convexity (concavity) with respect to the parameter. Corollaries from these results are applied to describe the properties of a conditionally optimal family. The influence functions of a number of conditionally optimal estimators for the shift and scale parameters of the normal model are constructed. The characteristics of these estimators are studied. The stability of most of the considered estimators is shown, which is important for their practical application. The theoretical results obtained can be useful in studying the properties of compromise estimators based on two criteria as well as in studying minimax contamination levels within the framework of A.M. Shurygin’s point Bayesian contamination model. The results of the work can be used in situations of purposed data corruption by an adversary including the problems related to adversarial machine learning.

It is shown that higher-order influence functions for M-estimators are mathematically equivalent to higher-order stochastic approximations to these estimators. The stochastic expansions are also shown to have corresponding higher-order U-statistic representations, providing an alternative approach for deriving and analyzing the approximate properties of M-estimators. [ABSTRACT FROM AUTHOR]

A generic out-of-sample error estimate is proposed for |$M$| -estimators regularized with a convex penalty in high-dimensional linear regression where |$(\boldsymbol{X},\boldsymbol{y})$| is observed and the dimension |$p$| and sample size |$n$| are of the same order. The out-of-sample error estimate enjoys a relative error of order |$n^{-1/2}$| in a linear model with Gaussian covariates and independent noise, either non-asymptotically when |$p/n\le \gamma $| or asymptotically in the high-dimensional asymptotic regime |$p/n\to \gamma ^{\prime}\in (0,\infty)$|⁠. General differentiable loss functions |$\rho $| are allowed provided that the derivative of the loss is 1-Lipschitz; this includes the least-squares loss as well as robust losses such as the Huber loss and its smoothed versions. The validity of the out-of-sample error estimate holds either under a strong convexity assumption, or for the L1-penalized Huber M-estimator and the Lasso under a sparsity assumption and a bound on the number of contaminated observations. For the square loss and in the absence of corruption in the response, the results additionally yield |$n^{-1/2}$| -consistent estimates of the noise variance and of the generalization error. This generalizes, to arbitrary convex penalty and arbitrary covariance, estimates that were previously known for the Lasso. [ABSTRACT FROM AUTHOR]

Estimating location is a central problem in functional data analysis, yet most current estimation procedures either unrealistically assume completely observed trajectories or lack robustness with respect to the many kinds of anomalies one can encounter in the functional setting. To remedy these deficiencies we introduce the first class of optimal robust location estimators based on discretely sampled functional data. The proposed method is based on M‐type smoothing spline estimation with repeated measurements and is suitable for both commonly and independently observed trajectories that are subject to measurement error. We show that under suitable assumptions the proposed family of estimators is minimax rate optimal both for commonly and independently observed trajectories and we illustrate its highly competitive performance and practical usefulness in a Monte‐Carlo study and a real‐data example involving recent Covid‐19 data. [ABSTRACT FROM AUTHOR]

Privacy concern in data sharing especially for health data gains particularly increasing attention nowadays. Now some patients agree to open their information for research use, which gives rise to a new question of how to effectively use the public information to better understand the private dataset without breaching privacy. In this paper, we specialize this question as selecting an optimal subset of the public dataset for M-estimators in the framework of differential privacy (DP) in [1]. From a perspective of non-interactive learning, we first construct the weighted private density estimation from the hybrid datasets under DP. Along the same line as [2], we analyze the accuracy of the DP M-estimators based on the hybrid datasets. Our main contributions are (i) we find that the bias-variance tradeoff in the performance of our M-estimators can be characterized in the sample size of the released dataset; (2) based on this finding, we develop an algorithm to select the optimal subset of the public dataset to release under DP. Our simulation studies and application to the real datasets confirm our findings and set a guideline in the real application.

M-type smoothing splines are a broad class of spline estimators that include the popular least-squares smoothing spline but also spline estimators that are less susceptible to outlying observations and model misspecification. However, available asymptotic theory only covers smoothing spline estimators based on smooth objective functions and consequently leaves out frequently used resistant estimators such as quantile and Huber-type smoothing splines. We provide a general treatment in this paper and, assuming only the convexity of the objective function, show that the least-squares (super-)convergence rates can be extended to M-type estimators whose asymptotic properties have not been hitherto described. We further show that auxiliary scale estimates may be handled under significantly weaker assumptions than those found in the literature and we establish optimal rates of convergence for the derivatives, which have not been obtained outside the least-squares framework. A simulation study and a real-data example illustrate the competitive performance of non-smooth M-type splines in relation to the least-squares spline on regular data and their superior performance on data that contain anomalies. [ABSTRACT FROM AUTHOR]

A confidence distribution is a distribution for a parameter of interest based on a parametric statistical model. As such, it serves the same purpose for frequentist statisticians as a posterior distribution for Bayesians, since it allows to reach point estimates, to assess their precision, to set up tests along with measures of evidence, to derive confidence intervals, comparing the parameter of interest with other parameters from other studies, etc. A general recipe for deriving confidence distributions is based on classical pivotal quantities and their exact or approximate distributions. However, in the presence of model misspecifications or outlying values in the observed data, classical pivotal quantities, and thus confidence distributions, may be inaccurate. The aim of this paper is to discuss the derivation and application of robust confidence distributions. In particular, we discuss a general approach based on the Tsallis scoring rule in order to compute a robust confidence distribution. Examples and simulation results are discussed for some problems often encountered in practice, such as the two-sample heteroschedastic comparison, the receiver operating characteristic curves and regression models. [ABSTRACT FROM AUTHOR]

Performance of any control scheme in Phase II depends directly on the quality of estimators utilized in Phase I. In practice, outliers could be present in the data which would impact the performance of estimators adversely. This study deals with robust parameter estimation and monitoring linear profiles in the presence of outliers and compares the results with the least squares (LS) estimators. For this purpose, M-estimators are used as robust estimators and empirical distributions for related statistics are determined using Mont Carlo simulation to calculate control limits for two T 2 control charts and for codding independent variable method. Using a numerical example, profile parameters are estimated by ordinary least squares and M-estimators and the resulting statistics are monitored by two T 2 control schemes. Phase II control charts are determined based on the two types of estimators and compared for different out of control profiles. Empirical distributions did not follow their exact distributions obtained by least squares method. Simulation results confirm that M-estimators lead to better estimates in comparison to LS estimators and also improves classification performance. Robust estimators also lead to improvement in ARL performance in comparison to LS estimators. [ABSTRACT FROM AUTHOR]

In this paper, a robust analysis of SO2 concentration measurements taken at the Belisario air quality monitoring station, Quito, Ecuador is carried out. The analyzed data contain 12 years of measurements, from January 2008 to December 2019. In addition, this set of measurements was decomposed into variables that represent each year, month, day of the week, and hour of the day in groups of two hours. For the analysis, classic, nonparametric and robust statistical methods were used, and the data were classified based on criteria established by the Quiteño Air Quality Index, taking confidence intervals into account. The results showed that the level of air pollution at the Belisario station due to the SO2 concentration is acceptable. In addition, the trend in the level of SO2 concentration decreased over the years studied, with a sharp drop from 2008 to 2012, then a small rise in 2013 and another fall until 2019, presenting decreasing oscillations that tend toward a desirable level of pollution. In this paper, it was shown that the air pollution at the Belisario station due to the concentration of SO2 in the last 12 years is not harmful to humans, with the measurement precision provided by robust statistical methods. Therefore, it can be concluded that the measures that have been taken by the Quito city council over the last few years are yielding good results.

Ridge regression is used to circumvent the problem of multicollinearity in the multiple linear regression models. Beside the multicollinearity, when the outliers in the y-direction are also present, then the usual ridge regression estimators gives inefficient results in terms of mean squared error (MSE). In order to mitigate such situation, ridge M-estimators are often used. Several estimators are available in literature but they do not perform well in terms of MSE when the joint problem of high multicollinearity and y-direction outliers is present. In this article, some new quantile based ridge M-estimators are proposed. The new estimators are then compared with other existing estimators through extensive Monte Carlo simulations for various error term distributions, degrees of multicollinearity and percentage of y-direction outliers. Based on simulation study with minimum MSE criterion, the new estimators outperform in many considered scenarios. Particularly, in case of high multicollinearity, y-direction outliers and heavy tailed error distributions, the proposed estimators have shown efficient results. A numerical example is also presented to support the simulation results. [ABSTRACT FROM AUTHOR]

Outliers in a statistical analysis strongly affect the performance of the ordinary least squares, such outliers need to be detected and extreme outliers deleted. This paper is aimed at proposing a redescending M-estimator, which is more efficient and robust, compared to other existing redescending M-estimators. The proposed method is applied to real life data to verify its effectiveness in detecting and deleting of outliers. The Monte Carlo simulation method is also used to investigate the performance of the newly proposed method. The results from the real life data and the Monte Carlo simulation method show that the proposed method is effective in the detection and deletion of extreme outliers compared to other existing redescending M-estimators. [ABSTRACT FROM AUTHOR]

Differential privacy is a cryptographically motivated approach to privacy that has become a very active field of research over the last decade in theoretical computer science and machine learning. In this paradigm, one assumes there is a trusted curator who holds the data of individuals in a database and the goal of privacy is to simultaneously protect individual data while allowing the release of global characteristics of the database. In this setting, we introduce a general framework for parametric inference with differential privacy guarantees. We first obtain differentially private estimators based on bounded influence M-estimators by leveraging their gross-error sensitivity in the calibration of a noise term added to them to ensure privacy. We then show how a similar construction can also be applied to construct differentially private test statistics analogous to the Wald, score, and likelihood ratio tests. We provide statistical guarantees for all our proposals via an asymptotic analysis. An interesting consequence of our results is to further clarify the connection between differential privacy and robust statistics. In particular, we demonstrate that differential privacy is a weaker stability requirement than infinitesimal robustness, and show that robust M-estimators can be easily randomized to guarantee both differential privacy and robustness toward the presence of contaminated data. We illustrate our results both on simulated and real data. for this article are available online. [ABSTRACT FROM AUTHOR]

This paper shows that least-square estimation (mean calculation) in a reproducing kernel Hilbert space (RKHS) F corresponds to different M-estimators in the original space depending on the kernel function associated with F. In particular, we present a proof of the correspondence of mean estimation in an RKHS for the Gaussian kernel with robust estimation in the original space performed with the Welsch M-estimator. This result is generalized to other types of M-estimators. This generalization facilitates the definition of new robust kernels associated to Huber, Tukey, Cauchy and Andrews M-estimators. The new kernels are empirically evaluated in different clustering tasks where state-of-the-art robust clustering methods are compared to kernel-based clustering using robust kernels. The results show that some robust kernels perform on a par with the best state-of-the-art robust clustering methods. [ABSTRACT FROM AUTHOR]

A highly popular regularized (shrinkage) covariance matrix estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward the grand mean of the eigenvalues of the SCM. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adaptive method to compute the optimal shrinkage parameter with minimum mean squared error is proposed. Our approach permits the use of any weight function such as Gaussian, Huber's, Tyler's, or t weight functions, all of which are commonly used in M-estimation framework. Our simulation examples illustrate that shrinkage M-estimators based on the proposed optimal tuning combined with robust weight function do not loose in performance to shrinkage SCM estimator when the data is Gaussian, but provide significantly improved performance when the data is sampled from an unspecified heavy-tailed elliptically symmetric distribution. Also, real-world and synthetic stock market data validate the performance of the proposed method in practical applications. [ABSTRACT FROM AUTHOR]

Recently, 5G networks have emerged as a new technology that can control the advancement of telecommunication networks and transportation systems. Furthermore, 5G networks provide better network performance while reducing network traffic and complexity compared to current networks. Machine-learning techniques (ML) will help symmetric IoT applications become a significant new data source in the future. Symmetry is a widely studied pattern in various research areas, especially in wireless network traffic. The study of symmetric and asymmetric faults and outliers (anomalies) in network traffic is an important topic. Nowadays, deep learning (DL) is an advanced approach in challenging wireless networks such as network management and optimization, anomaly detection, predictive analysis, lifetime value prediction, etc. However, its performance depends on the efficiency of training samples. DL is designed to work with large datasets and uses complex algorithms to train the model. The occurrence of outliers in the raw data reduces the reliability of the training models. In this paper, the performance of Vehicle-to-Everything (V2X) traffic was estimated using the DL algorithm. A set of robust statistical estimators, called M-estimators, have been proposed as robust loss functions as an alternative to the traditional MSE loss function, to improve the training process and robustize DL in the presence of outliers. We demonstrate their robustness in the presence of outliers on V2X traffic datasets.

In many real cases the data are not expressed in term of single values but are imprecise. In all these cases, standard clustering methods for single-valued data are unable to properly take into account the imprecise nature of the data. In this paper, by considering the Partitioning Around Medoids (PAM) approach in a fuzzy framework, we propose a fuzzy clustering method for imprecise data formalized in a fuzzy manner. In particular, in order to neutralize the negative effects of possible outlier fuzzy data in the clustering process, we proposed a robust fuzzy c-medoids clustering method for fuzzy data based on the combination of Huber's M-estimators and Yager's OWA (Ordered Weighted Averaging) operators. The proposed method is able to smooth the influence of anomalous data by means of a suitable parameter, the so-called typicality parameter, capable to tune the influence of the outliers. The performance of the proposed method has been shown by means of a simulation study, composed of experiments on: (i) simple two-dimensional dataset, (ii) benchmark datasets and (iii) the fuzzy-art-outliers dataset. The comparison made with the robust clustering methods known from the literature indicates the competitiveness of the introduced method to others. An application of the suggested method to a real dataset is also provided and the results of the method has been compared with other clustering methods suggested in the literature. In the application, the comparative assessment has shown the informational gain (in term of additional information) of the proposed method vs the other robust methods. [ABSTRACT FROM AUTHOR]

Modern techniques for the map analysis allow for the creation of full or partial geometric reconstruction of its content. The projection is described by the set of estimated constant values: transformed pole position, standard parallel latitude, longitude of the central meridian, and a constant parameter. Analogously the analyzed map is represented by its constant values: auxiliary sphere radius, origin shifts, and angle of rotation. Several new methods denoted as M6-M9 for the estimation of an unknown map projection and its parameters differing in the number of determined parameters, reliability, robustness, and convergence have been developed. However, their computational demands are similar. Instead of directly measuring the dissimilarity of two projections, the analyzed map in an unknown projection and the image of the sphere in the well-known (i.e., analyzed) projection are compared. Several distance functions for the similarity measurements based on the location as well as shape similarity approaches are proposed. An unconstrained global optimization problem poorly scaled, with large residuals, for the vector of unknown parameters is solved by the hybrid BFGS method. To avoid a slower convergence rate for small residual problems, it has the ability to switch between first- and second-order methods. Such an analysis is beneficial and interesting for historic, old, or current maps without information about the projection. Its importance is primarily referred to refinement of spatial georeference for the medium- and small-scale maps, analysis of the knowledge about the former world, analysis of the incorrectly/inaccurately drawn regions, and appropriate cataloging of maps. The proposed algorithms have been implemented in the new version of the detectproj software.

To obtain M-estimators of a response variable when the data are missing at random, we can construct three bias-corrected nonparametric estimating equations based on inverse probability weighting, mean imputation, and augmented inverse probability weighting approaches. However, when the dimension of covariate is not low, the estimation efficiency will be affected due to the curse of dimensionality. To address this issue, we propose a two-stage estimation procedure by using the dimension-reduced kernel estimators in conjunction with bias-corrected estimating equations. We show that the resulting three kernel-assisted estimating equations yield asymptotically equivalent M-estimators that achieve the desirable properties. The finite-sample performance of the proposed estimators for response mean, distribution function and quantile is studied through simulation, and an application to HIV-CD4 data set is also presented. [ABSTRACT FROM AUTHOR]

This article develops empirical likelihood methodology for a class of long range dependent processes driven by a stationary Gaussian process. We consider population parameters that are defined by estimating equations in the time domain. It is shown that the standard block empirical likelihood (BEL) method, with a suitable scaling, has a non‐standard limit distribution based on a multiple Wiener–Itô integral. Unlike the short memory time series case, the scaling constant involves unknown population quantities that may be difficult to estimate. Alternative versions of the empirical likelihood method, involving the expansive BEL (EBEL) methods are considered. It is shown that the EBEL renditions do not require an explicit scaling and, therefore, remove this undesirable feature of the standard BEL. However, the limit law involves the long memory parameter, which may be estimated from the data. Results from a moderately large simulation study on finite sample properties of tests and confidence intervals based on different empirical likelihood methods are also reported. [ABSTRACT FROM AUTHOR]

M-estimators offer simple robust alternatives to the maximum likelihood estimator. The density power divergence (DPD) and the logarithmic density power divergence (LDPD) measures provide two classes of robust M-estimators which contain the MLE as a special case. In each of these families, the robustness of the estimator is achieved through a density power down-weighting of outlying observations. Even though the families have proved to be useful in robust inference, the relation and hierarchy between these two families are yet to be fully established. In this paper, we present a generalized family of divergences that provides a smooth bridge between DPD and LDPD measures. This family helps to clarify and settle several longstanding issues in the relation between the important families of DPD and LDPD, apart from being an important tool in different areas of statistical inference in its own right. [ABSTRACT FROM AUTHOR]

This letter proposes a new method for polarimetric synthetic aperture radar (PolSAR) denoising. More precisely, it seeks to address a new statistical approach for weights computation in nonlocal (NL) approaches. The aim is to present a simple criterion using $M$ -estimators and to detect similar pixels in an image. A binary hypothesis test is used to select similar pixels which will be used for covariance matrix estimation together with associated weights. The method is then compared with an advanced state-of-the-art PolSAR denoising method named NL-SAR. The filter performances are measured by a set of different indicators, including relative errors on incoherent target decomposition parameters, coherences, polarimetric signatures, and edge preservation on a set of simulated PolSAR images. Finally, results for RADARSAT-2 PolSAR data are presented. [ABSTRACT FROM AUTHOR]

Abstract Archetypal analysis approximates data by means of mixtures of actual extreme cases (archetypoids) or archetypes, which are a convex combination of cases in the data set. Archetypes lie on the boundary of the convex hull. This makes the analysis very sensitive to outliers. A robust methodology by means of M-estimators for classical multivariate and functional data is proposed. This unsupervised methodology allows complex data to be understood even by non-experts. The performance of the new procedure is assessed in a simulation study, where a comparison with a previous methodology for the multivariate case is also carried out, and our proposal obtains favorable results. Finally, robust bivariate functional archetypoid analysis is applied to a set of companies in the S&P 500 described by two time series of stock quotes. A new graphic representation is also proposed to visualize the results. The analysis shows how the information can be easily interpreted and how even non-experts can gain a qualitative understanding of the data. Highlights • Robust archetypal analysis for multivariate data and time series. • Clustering financial time series. • Functional data analysis. • Robust statistics. • Outlier or anomaly detection. [ABSTRACT FROM AUTHOR]

This paper deals with the asymptotic statistical properties of a class of redescending M-estimators in linear models with increasing dimension. This class is large enough to include popular high breakdown point estimators such as Sestimators and MM-estimators, which were not covered by existing results in the literature. We prove consistency assuming only that p=n ! 0 and asymptotic normality essentially if p3/n 0, where p is the number of covariates and n is the sample size. [ABSTRACT FROM AUTHOR]

In this study, two new types of estimators of the location and scale parameters are proposed having high efficiency and robustness; the dynamically weighted modified maximum likelihood (DWMML) and the combined dynamically weighted modified maximum likelihood (CDWMML) estimators. Three pairs of the DWMML and two pairs of the CDWMML estimators of the location and scale parameters are produced, namely, the DWMML1, the DWMML2 and the DWMML3, and the CDWMML1 and the CDWMML2 estimators, respectively. Based on the simulation results, the DWMML1 estimators of the location and scale parameters are almost fully efficient (under normality) and robust at the same time. The DWMML3 estimators are asymptotically fully efficient and more robust than the M-estimators. The DWMML2 estimators are a compromise between efficiency and robustness. The CDWMML1 and CDWMML2 estimators are jointly very efficient and robust. Particularly, the CDWMML1 and CDWMML2 estimators of the scale parameter are superior compared to the other estimators of the scale parameter. [ABSTRACT FROM AUTHOR]

This paper introduces a weighted Z-estimator for moment condition models, assuming auxiliary information on the unknown distribution of the data and under the assumption of weak dependence (strong mixing processes). We model serial dependence through a simple nonparametric blocking device, routinely used in the bootstrap literature. The weights that carry the auxiliary information are computed by means of generalized empirical likelihood. The resulting weighted estimator is shown to be consistent and asymptotically normal. The proposed estimator is computationally simple and shows nice finite sample features when compared to asymptotically equivalent estimators. [ABSTRACT FROM AUTHOR]

In this study we compared the efficiency and robustness of several estimators, namely, the least squares (LS) estimators, the Huber and Tukey Mestimators, the S-estimators and the MM-estimators for the parameters of the general linear regression (GLR) model via simulation. First, the programs for each method were written by using Matlab. Then, an extensive simulation study was conducted under several models. The results are consistent with the literature but some important points were also found to be remarked. As the literature suggests, in general, the MM-estimators are the most efficient estimators, and among the robust estimators discussed here, the S-estimators are the least efficient ones. Naturally, the LS estimators are badly affected by the deviations from the assumed model because of their sensitive nature. Moreover, it was found that while the LS estimator of the variance of the error term is unbiased, the robust estimators discussed here are generally biased. Additionally, the MM-estimator of the variance of the error term is less biased than the other robust estimators and its bias gets smaller faster as the sample size increases compared to the others. At the end of the study, to be more illustrative, two real life data examples were given with the related comments. [ABSTRACT FROM AUTHOR]

In many applications involving large dataset or online learning, stochastic gradient descent (SGD) is a scalable algorithm to compute parameter estimates and has gained increasing popularity due to its numerical convenience and memory efficiency. While the asymptotic properties of SGD-based estimators have been well established, statistical inference such as interval estimation remains much unexplored. The classical bootstrap is not directly applicable if the data are not stored in memory. The plug-in method is not applicable when there is no explicit formula for the covariance matrix of the estimator. In this paper, we propose an online bootstrap procedure for the estimation of confidence intervals, which, upon the arrival of each observation, updates the SGD estimate as well as a number of randomly perturbed SGD estimates. The proposed method is easy to implement in practice. We establish its theoretical properties for a general class of models that includes linear regressions, generalized linear models, M-estimators and quantile regressions as special cases. The finite-sample performance and numerical utility is evaluated by simulation studies and real data applications. [ABSTRACT FROM AUTHOR]

The divide and conquer method is a common strategy for handling massive data. In this article, we study the divide and conquer method for cubic-rate estimators under the massive data framework. We develop a general theory for establishing the asymptotic distribution of the aggregated M-estimators using a weighted average with weights depending on the subgroup sample sizes. Under certain condition on the growing rate of the number of subgroups, the resulting aggregated estimators are shown to have faster convergence rate and asymptotic normal distribution, which are more tractable in both computation and inference than the original M-estimators based on pooled data. Our theory applies to a wide class of M-estimators with cube root convergence rate, including the location estimator, maximum score estimator, and value search estimator. Empirical performance via simulations and a real data application also validate our theoretical findings. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Generalized linear mixed models are powerful tools for analyzing clustered data, where the unknown parameters are classically (and most commonly) estimated by the maximum likelihood and restricted maximum likelihood procedures. However, since the likelihood-based procedures are known to be highly sensitive to outliers, M-estimators have become popular as a means to obtain robust estimates under possible data contamination. In this paper, we prove that for sufficiently smooth general loss functions defining the M-estimators in generalized linear mixed models, the tail probability of the deviation between the estimated and the true regression coefficients has an exponential bound. This implies an exponential rate of consistency of these M-estimators under appropriate assumptions, generalizing the existing exponential consistency results from univariate to multivariate responses. We have illustrated this theoretical result further for the special examples of the maximum likelihood estimator and the robust minimum density power divergence estimator, a popular example of model-based M-estimators, in the settings of linear and logistic mixed models, comparing it with the empirical rate of convergence through simulation studies. [ABSTRACT FROM AUTHOR]

Ochroma pyramidale (Cav. ex. Lam.) Urb. (balsa-tree) is a commercially important tree species that ranges from Mexico to northern Brazil. Due to its low weight and mechanical endurance, the wood is particularly well-suited for wind turbine blades, sporting equipment, boats and aircrafts; as such, it is in high market demand and plays an important role in many regional economies. This tree species is also well-known to exhibit a high degree of variation in growth. Researchers interested in modeling the height−diameter relationship typically resort to using ordinary least squares (OLS) to fit linear models; however, this method is known to suffer from sensitivity to outliers. Given the latter, the application of these models may yield potentially biased tree height estimates. The use of robust regression with iteratively reweighted least squares (IRLS) has been proposed as an alternative to mitigate the influence of outliers. This study aims to improve the modeling of height−diameter relationships of tree species with high growth variation, by using robust regressions with IRLS for data-sets stratified by site-index and age-classes. We implement a split sample approach to assess the model performance using data from Ecuador’s continuous forest inventory (n = 32,279 trees). A sensitivity analysis of six outlier scenarios is also conducted using a subsample of the former (n = 26). Our results indicate that IRLS regression methods can give unbiased height predictions. At face value, the sensitivity analysis indicates that OLS performs better in terms of standard error of estimate. However, we found that OLS suffers from skewed residual distributions (i.e., unreliable estimations); conversely, IRLS seems to be less affected by this source of bias and the fitted parameters indicate lower standard errors. Overall, we recommend using robust regression methods with IRLS to produce consistent height predictions for O. pyramidale and other tree species showing high growth variation.

We study the asymptotic behavior of one-step weighted M-estimators based on independent not necessarily identically distributed observations, which approximate consistent weighted M-estimators. We find sufficient conditions for asymptotic normality of these estimators. As an application, we consider some known regression models where the one-step estimation under consideration allows us to construct explicit asymptotically optimal estimators having the same accuracy as the least-squares or quasi-likelihood estimators. [ABSTRACT FROM AUTHOR]

-->Highlights • Developed extended Kalman filter and moving window estimator that are robust w.r.t. gross errors in measurements. • Use of M-estimators to achieve robustness. • Demonstrated simultaneous state and parameter/disturbance estimation in the presence of measurement gross errors. Abstract Simultaneous occurrence of gross errors (outliers/biases/drifts) in the measured signals, and drifting disturbances/parameter variations affecting the system dynamics can lead to biased state estimates, and, in turn, can lead to deterioration in the performance of model-based monitoring and control schemes. In this work, robust recursive and moving window based Bayesian state and parameter estimators are developed that are robust w.r.t. gross errors in the measurements and can simultaneously estimate non-additive unmeasured disturbance/parameter variations. Using Bayes’ rule, the update step of Kalman filter (KF) is recast as an optimization problem. The optimization is then modified by replacing the likelihood term in the objective function with cost function defined by an M-estimator. The M-estimators considered in this work are Huber's Fair function and Hampel's redescending estimator. The reformulated KF is then used as a basis for reformulating extended Kalman filter (EKF). This re-formulated EKF is then used for developing robust simultaneous state and parameter estimation schemes. In particular, a robust version of recently proposed moving window based state and parameter estimator [1] has been developed. The resulting formulation can be viewed as a hybrid approach, in which the gross errors in the measurements are dealt with in a passive manner, with an active elimination of model plant mismatch by estimating unmeasured disturbance/parameter variations simultaneously. The efficacy of the proposed robust state and parameter estimators is demonstrated by conducting simulation studies and experimental studies. Analysis of the simulation and experimental results reveal that the proposed robust recursive and moving window based state and parameter estimators significantly reduce or completely nullify the effect of gross errors on the state estimates while simultaneously estimating drifting unmeasured disturbances/parameters. The simulation study also underscores the importance of simultaneous estimation of unmeasured disturbances/parameters while achieving robustness using the M-estimators. Moreover, Hampel's redescending estimator is found to be a better choice of M-estimator than the popular Huber's Fair function, as the redescending estimator can completely nullify the effect of gross errors on the state and parameter estimates. [ABSTRACT FROM AUTHOR]

The kernel recursive least squares (KRLS), a nonlinear counterpart of the famed RLS algorithm, performs linear regression in a high-dimensional feature space induced by a Mercer kernel. Despite the growing interest in the KRLS for nonlinear signal processing, the presence of outliers in the estimation data causes the resulting predictor's performance to deteriorate considerably. Bearing this in mind, we introduce an approach to amalgamate the kernel-based learning framework that gives rise to the KRLS algorithm with the robust regression framework of M-estimators with the aim of building an outlier-robust variant for the KRLS. Initially, we develop the theoretical aspects of the proposed algorithm and then analyze its behavior in nonlinear system identification problems using synthetic and real-world datasets (including a large-scale one) contaminated with outliers. The obtained results indicate that the robust variant of the KRLS algorithm consistently outperforms the state-of-the-art in robust adaptive filtering algorithms, as the amount of outliers in the data increases. [ABSTRACT FROM AUTHOR]