Search

This chapter provides a brief and intuitive summary of the econometric theory that underlies OLS estimation, including: The data generating process and sample regression functions Types of data Ordinary Least Squares (OLS) estimation Measuring correlation and goodness of fit The Gauss–Markov theorem Properties of estimators Hypothesis testing Introduction Running Regressions is about the quantitative analysis of observed behaviours and phenomena using economic theory, probability and statistics. By bringing these different elements together we can improve our understanding of people, firms and countries – in the past, the present and the future. For those readers who approach the subject with trepidation, it may be worth remembering this practical goal. The skills that are required are not intrinsically difficult if a systematic approach is followed. Not least, our intention in Running Regressions is to illustrate commonly used techniques in an imaginative and interesting way. This chapter provides a brief introduction to concepts and techniques that are then worked through as practical examples in the following chapters. Finally, whilst we focus on economics, finance and development studies in our selection of topics, the approach can be used in analysing a wide range of all real-world situations. The techniques can be, and are, used in all the social and natural sciences. Models and data The aim of econometric analysis is to understand the data generating processes (DGPs) that underlie economic systems and human behaviour. These DGPs are like the engines in a car and they propel socio-economic actions and events.

[J]ust a year after getting his degree, Mansfield Merriman, Ph.D. Yale 1876, wrote of Gauss's elegant demonstration of the “Gauss-Markov” theorem that “The proof is entirely untenable.” In charity to Merriman it might be added that no less a mathematician than Poincare also misconstrued the nature of Gauss's result. Stephen M. Stigler Introduction This chapter brings together all the key ideas in this book: In order to do inference one must have a model of the data generating process. There are many possible estimators of the population parameters. Estimators can be classified according to whether they are unbiased – that is, on average correct. Many, but by no means all, estimators are linear estimators. One of the main criteria for comparing estimators is the variance of the estimator. When the data are generated according to the classical econometric box model, ordinary least squares is the best estimator in the class of linear, unbiased estimators – best, that is, according to the criterion of finding the estimator with the minimum variance for a given sample size. This last statement is often stated in shorthand as “OLS is BLUE” (best linear unbiased estimator) and is known as the Gauss–Markov theorem from which the title of this chapter is derived. This theorem explains the preeminence of the OLS estimator in econometrics. The Gauss–Markov theorem also works in reverse: when the data generating process does not follow the classical econometric model, ordinary least squares is typically no longer the preferred estimator.

A geometrical setting is constructed, based on Hilbert space, in which the asymptotic properties of estimators can be studied. Estimators are dened in the context