Search

The article revisits univariate and multivariate linear regression models. It is shown that least-square estimators (LSEs) are minimum risk estimators in general class of linear unbiased estimators...

In this chapter, mean-square linear estimation is discussed and the normal equations are derived. The orthogonality theorem concerning random variables is introduced as an alternative for their derivation. Issues concerning complex random variables, such as widely-linear estimation and Wirtinger’s calculus are presented. Some typical applications of MSE, such as image deblurring, interference cancellation, system identification and channel equalization are defined. Issues related to the efficient solution of the normal equations in the context of linear filtering are discussed and the Levinson and lattice-ladder algorithms are derived. The Gauss-Markov theorem for MSE of linear models is provided and the case of constrained estimation in the context of beamforming is introduced. The chapter concludes with the introduction and derivation of Kalman filtering.

Algebraic proofs of Gauss–Markov theorem are very disappointing from an intuitive point of view. An alternative is to use geometry that emphasizes the essential statistical ideas behind the result. This article presents a truly geometrical intuitive approach to the theorem, based only in simple geometrical concepts, like linear subspaces and orthogonal projections.

Observations which are realizations from some continuous process are frequent in sciences, engineering, economics, and other fields. We consider linear models, with possible random effects, where the responses are random functions in a suitable Sobolev space. The processes cannot be observed directly. With smoothing procedures from the original data, both the response curves and their derivatives can be reconstructed, even separately. From both these samples of functions, just one sample of representatives is obtained to estimate the vector of functional parameters. A simulation study shows the benefits of this approach over the common method of using information either on curves or derivatives. The main theoretical result is a strong functional version of the Gauss-Markov theorem. This ensures that the proposed functional estimator is more efficient than the best linear unbiased estimator based only on curves or derivatives., the best information from the two samples of functions and derivatives: a strong version of the Gauss-Markov theorem. Relaxed an hidden hypothesis on linear independence of the Riesz representation of the Karhunen-Loeve base

Pursuing abstractness and generality, a number of books and articles in econometrics rely heavily on standard algebraic proofs. However, most of the theorems proved in such a way have a strong geometric appeal. This paper demonstrates how shorter, less technical and even more beautiful the proofs can be when they are based on geometric theorems. We show that this technique can be extended to explain concepts in probability and statistics. Although most of the theorems and ideas are not completely new and there are even a few works on the geometry in econometrics and statistics, this paper introduces alternative explanations and more general results in some cases. Further, there are algebraic proofs provided parallel to geometric ones and illustrations which are our own work.

The present article is devoted mainly to methods of small-molecule structure refinement against single-crystal diffraction data, offered by widely distributed software and applied to chemical crystallography. The theory of least-squares refinement and maximum likelihood, the use of constraints and restraints, the weighting of the observations, and the estimation of uncertainties of the results are presented.

The 1997 Biometrics paper by Mike Kenward and James Roger has become a citation classic (more than 1260 citations by End June 2013 according to Google Scholar) and the solution that they proposed to deal with the problem of significance tests of fixed effects in REML models is now incorporated in many software packages and accepted by all biostatisticians as the method of choice. Nevertheless, it does not solve all problems, since there is more to analysis than just significance and since the problems that models with more than one variance pose arise in many contexts. In this paper, I discuss some problems and applications and make some tentative suggestions as to how they may be tackled. My excuse for raising problems I do not solve is that it may inspire James and Mike to complete what they started.

The basic mathematical principle for bundle adjustment (BA) in photogrammetry is the Gauss-Markov Theorem within the framework of classical statistical inference. In the present article we try to show how Bayesian statistics can be applied in this field, leading to a so-called Bayesian bundle adjustment. The rigorous implementation of the Bayesian approach is derived and a comparison with the traditional BA both in theory and practice is accomplished. The empirical test results show that the Bayesian approach achieves almost the same accuracy as the conventional one, but with advantages as well as some difficulties to be discussed at the end of this paper. © 2011 Published by Elsevier Ltd.

For the linear regression with AR(1) errors model, the robust generalized and feasible generalized estimators of Lai et al. (2003) of regression parameters are shown to have the desired property of a robust Gauss Markov theorem. This is done by showing that these two estimators are the best among classes of linear trimmed means. Monte Carlo and data analysis for this technique have been performed.

This paper presents a method of estimating the regression and variance parameters in the multiple linear regression Berkson model for a continuous-time stochastic process with uncorrelated increments. Under minimal conditions, we establish (i) the Gauss–Markov theorem and the quadratic mean—as well as the strong consistency of the proposed estimate of the regression parameter and (ii) the weak consistency of the proposed estimate of the variance parameter.

Stratifying and matching by the propensity score are increasingly popular approaches to deal with confounding in medical studies investigating effects of a treatment or exposure. A more traditional alternative technique is the direct adjustment for confounding in regression models. This paper discusses fundamental differences between the two approaches, with a focus on linear regression and propensity score stratification, and identifies points to be considered for an adequate comparison. The treatment estimators are examined for unbiasedness and efficiency. This is illustrated in an application to real data and supplemented by an investigation on properties of the estimators for a range of underlying linear models. We demonstrate that in specific circumstances the propensity score estimator is identical to the effect estimated from a full linear model, even if it is built on coarser covariate strata than the linear model. As a consequence the coarsening property of the propensity score-adjustment for a one-dimensional confounder instead of a high-dimensional covariate-may be viewed as a way to implement a pre-specified, richly parametrized linear model. We conclude that the propensity score estimator inherits the potential for overfitting and that care should be taken to restrict covariates to those relevant for outcome.

Carl Friedrich Gauss was born on 30 April 1777 in Brunswick (Germany) and died in Gottingen (Germany) on 23 February 1855. Gauss contributed to all areas of science known in his day, notably to astronomy, physics, and geodesy, but this article concentrates on his contributions to statistics and numerical methods. In 1805, Legendre published an algebraic procedure for the linear fitting problem that he named the method of least squares. Gauss's first (1809) statistical derivation of this procedure was based on a combination of the normal distribution with a Bayesian variant of the maximum likelihood optimality criterion. Later, Gauss (1823–28) gave a very full account of an alternative decision-theoretic approach, which employed the mean squared error of the estimator as optimality criterion. In particular, he identified the least squares estimator as the linear unbiased estimator of the parameters with minimum mean squared error. Gauss developed the Gaussian elimination technique for solving determinate systems of linear equations, and explained how nonlinear relationships may be fitted by means of an iterative variant of the method of least squares. He also developed a second iterative (relaxation) procedure related to the Gauss–Seidel procedure.

The minimum variance linear unbiased estimators (MVLUE), the best linear invariant estimators (BLIE) and the maximum likelihood estimators (MLE) based on n-selected generalized order statistics are presented for the parameters of the Burr XII distribution.

In this paper, first we make a maximal extension of the well-known Gauss–Markov Theorem (GMT) in its linear framework. In particular, the maximal class of distributions of error term for which the GMT holds is derived. Second, we establish a nonlinear version of the maximal GMT and describe some interesting families of distributions of error term for which the nonlinear GMT holds.

The Gauss–Markov theorem states that, under very general conditions, which do not require Gaussian assumptions, the ordinary least squares method, in linear regression models, provides best linear unbiased estimators (BLUE), a property which constitutes the theoretical justification for that widespread estimation method. Keywords: linear regression; least square; BLUE estimation; maximum likelihood; gauss; legendre; markov

The Gauss–Markov theorem provides a golden standard for constructing the best linear unbiased estimation for linear models. The main purpose of this article is to extend the Gauss–Markov theorem to include nonparametric mixed-effects models. The extended Gauss–Markov estimation (or prediction) is shown to be equivalent to a regularization method and its minimaxity is addressed. The resulting Gauss–Markov estimation serves as an oracle to guide the exploration for effective nonlinear estimators adaptively. Various examples are discussed. Particularly, the wavelet nonparametric regression example and its connection with a Sobolev regularization is presented.

The ranked-set sampling technique has been generalized so that more efficient estimators may be obtained. The generalized ranked-set sampling technique is applied in the estimation of location and scale parameters of the half-logistic distribution. Three estimators are proposed. These are minimum variance unbiased estimators, simple estimators and ranked-set sample estimators. Coefficients, variances, covariances, generalized variances and relative efficiencies are tabulated. The estimators are compared to the best linear unbiased estimators of the location and scale parameters.

The minimum variance linear unbiased estimators (MVLUEs) of population parameters based on ranked set sampling are discussed for some distributions. Also values of the coefficients required for obtaining MVLUEs of the location and scale parameters are presented. For the same distributions, MVLUEs of the population mean are also obtained and the required coefficients are supplied. The MVLUEs of the population mean are compared with the other popular simple random sample (SRS) and ranked set sample (RSS) estimators of the population mean. The MVLUEs of the population mean are found to be more precise than the usual SRS or RSS estimators of the population mean for all cases considered.

The problem of estimating regression coefficients when the observations are obtained by pairs is considered in this article. Three estimation methods are introduced and compared: ordinary least squares by treating paired data as two separate observations (OLS), ordinary least squares by taking the averages of two correlated observations (AOLS), and generalized least squares by estimating the variance-covariance matrix of the error terms (EGLS). In general, OLS method yields a better estimate than that of AOLS. Simulation studies are conducted to determine the sample sizes required for the EGLS estimate to be better than that of OLS.

This note presents a generalised Gauss-Markov Theorem on least squares estimation and making use of the theorem, argues how to identify the least squares estimators that are to be the best linear unbiased estimates.

Linear regression amounts to estimating a linear model that maps features e.g., age or gender to corresponding data e.g., the answer to a survey or the outcome of a medical exam. It is a ubiquitous tool in experimental sciences. We study a setting in which features are public but the data is private information. While the estimation of the linear model may be useful to participating individuals, if, e.g., it leads to the discovery of a treatment to a disease, individuals may be reluctant to disclose their data due to privacy concerns. In this paper, we propose a generic game-theoretic model to express this trade-off. Users add noise to their data before releasing it. In particular, they choose the variance of this noise to minimize a cost comprising two components: a a privacy cost, representing the loss of privacy incurred by the release; and b an estimation cost, representing the inaccuracy in the linear model estimate. We study the Nash equilibria of this game, establishing the existence of a unique non-trivial equilibrium. We determine its efficiency for several classes of privacy and estimation costs, using the concept of the price of stability. Finally, we prove that, for a specific estimation cost, the generalized least-square estimator is optimal among all linear unbiased estimators in our non-cooperative setting: this result extends the famous Aitken/Gauss-Markov theorem in statistics, establishing that its conclusion persists even in the presence of strategic individuals.

If indices are to be used as the variables predicted by linear statistical models, it is important to be able to recover as much local information as possible from the values forecast for the indices. Here it is shown that the indices that encapsulate the most information about the local climatic state are determined by a generalized (two-matrix) eigenvalue problem that is equivalent to the usual (one-matrix) eigenvalue problem involving the sample correlation matrix. Thus, the best indices in the sense of providing the most location-specific information are familiar principal-component indices. Regarding the indices as predictors in linear statistical models similar to those routinely used for estimating meteorological fields from observations reveals the role of the Gauss-Markov theorem in EOF analyses. From this perspective each index can he characterized by two EOF-like maps: the first illustrating the linear combinations of the data used to define the index, and the second displaying the Gau...

The Gauss–Markov theorem states that, under very general conditions, which do not require Gaussian assumptions, the ordinary least squares method, in linear regression models, provides best linear unbiased estimators (BLUE), a property which constitutes the theoretical justification for that widespread estimation method. Keywords: linear regression; least square; BLUE estimation; maximum likelihood; gauss; legendre; markov

When stating the Gauss-Markov theorem, undergraduate econometric textbooks generally imply that the ordinary least squares (OLS) estimator has minimum variance. However, the proof of the Gauss Markov theorem indicates that the weights produced by the OLS estimator–not the formula per se–produce the unique minimum–variance estimator. An example demonstrates that other linear unbiased estimators can yield the same variance as the OLS estimator; however, the weights from such formulas are identical to the OLS weights. To avoid this pedagogical confusion, the statement of the theorem should emphasize the weights rather than the estimator itself.