Search

We study batched bandit experiments and consider the problem of inference conditional on the realized stopping time, assignment probabilities, and target parameter, where all of these may be chosen adaptively using information up to the last batch of the experiment. Absent further restrictions on the experiment, we show that inference using only the results of the last batch is optimal. When the adaptive aspects of the experiment are known to be location-invariant, in the sense that they are unchanged when we shift all batch-arm means by a constant, we show that there is additional information in the data, captured by one additional linear function of the batch-arm means. In the more restrictive case where the stopping time, assignment probabilities, and target parameter are known to depend on the data only through a collection of polyhedral events, we derive computationally tractable and optimal conditional inference procedures., Comment: An extended abstract of this paper was presented at CODE@MIT 2021

This paper studies optimal decision rules, including estimators and tests, for weakly identified GMM models. We derive the limit experiment for weakly identified GMM, and propose a theoretically‐motivated class of priors which give rise to quasi‐Bayes decision rules as a limiting case. Together with results in the previous literature, this establishes desirable properties for the quasi‐Bayes approach regardless of model identification status, and we recommend quasi‐Bayes for settings where identification is a concern. We further propose weighted average power‐optimal identification‐robust frequentist tests and confidence sets, and prove a Bernstein‐von Mises‐type result for the quasi‐Bayes posterior under weak identification.

In models with potentially weak identification, researchers often decide whether to report a robust confidence set based on an initial assessment of model identification. Two-step procedures of this sort can generate large coverage distortions for reported confidence sets, and existing procedures for controlling these distortions are quite limited. This paper introduces a generally applicable approach to detecting weak identification and constructing two-step confidence sets in GMM. This approach controls coverage distortions under weak identification and indicates strong identification, with probability tending to 1 when the model is well identified.

We propose a local measure of the relationship between parameter estimates and the moments of the data they depend on. Our measure can be computed at negligible cost even for complex structural models. We argue that reporting this measure can increase the transparency of structural estimates, making it easier for readers to predict the way violations of identifying assumptions would affect the results.When the key assumptions are orthogonality between error terms and excluded instruments, we show that our measure provides a natural extension of the omitted variables bias formula for nonlinear models. We illustrate with applications to published articles in several fields of economics. JEL Codes: C10, C52., National Science Foundation (U.S.)

We derive mean-unbiased estimators for the structural parameter in instrumental variables models with a single endogenous regressor where the sign of one or more first-stage coefficients is known. In the case with a single instrument, there is a unique nonrandomized unbiased estimator based on the reduced-form and first-stage regression estimates. For cases with multiple instruments we propose a class of unbiased estimators and show that an estimator within this class is efficient when the instruments are strong. We show numerically that unbiasedness does not come at a cost of increased dispersion in models with a single instrument: in this case the unbiased estimator is less dispersed than the two-stage least squares estimator. Our finite-sample results apply to normal models with known variance for the reduced-form errors, and imply analogous results under weak-instrument asymptotics with an unknown error distribution. Keyword: unbiased estimation; weak instruments

We consider how a firm dynamically allocates business among several suppliers to motivate them in a relational contract. The firm chooses one supplier who exerts private effort. Output is non-contractible, and each supplier observes only his own relationship with the principal. In this setting, allocation decisions constrain the transfers that can be promised to suppliers in equilibrium. Consequently, optimal allocation decisions condition on payoff-irrelevant past performance to make strong incentives credible. We construct a dynamic allocation rule that attains first-best whenever any allocation rule does. Thisallocation rule performs strictly better than any rule that depends only on payoff-relevant information. (JEL D21, D82, L14, L24).

Some empirical results are more likely to be published than others. Such selective publication leads to biased estimates and distorted inference. This paper proposes two approaches for identifying the conditional probability of publication as a function of a study’s results, the first based on systematic replication studies and the second based on meta-studies. For known conditional publication probabilities, we propose median-unbiased estimators and associated confidence sets that correct for selective publication. We apply our methods to recent large-scale replication studies in experimental economics and psychology, and to meta-studies of the effects of minimum wages and de-worming programs.

Conventional tests for composite hypotheses in minimum distance models can be unreliable when the relationship between the structural and reduced‐form parameters is highly nonlinear. Such nonlinearity may arise for a variety of reasons, including weak identification. In this note, we begin by studying the problem of testing a “curved null” in a finite‐sample Gaussian model. Using the curvature of the model, we develop new finite‐sample bounds on the distribution of minimum‐distance statistics. These bounds allow us to construct tests for composite hypotheses which are uniformly asymptotically valid over a large class of data generating processes and structural models., Massachusetts Institute of Technology (Castle-Krob Career Development Chair), Alfred P. Sloan Foundation (Sloan Research Fellowship)

This paper shows that the problem of testing hypotheses in moment condition models without any assumptions about identification may be considered as a problem of testing with an infinite‐dimensional nuisance parameter. We introduce a sufficient statistic for this nuisance parameter in a Gaussian problem and propose conditional tests. These conditional tests have uniformly correct asymptotic size for a large class of models and test statistics. We apply our approach to construct tests based on quasi‐likelihood ratio statistics, which we show are efficient in strongly identified models and perform well relative to existing alternatives in two examples.

Conventional tests for composite hypotheses in minimum distance models can be unreliable when the relationship between the structural and reduced‐form parameters is highly nonlinear. Such nonlinearity may arise for a variety of reasons, including weak identification. In this note, we begin by studying the problem of testing a “curved null” in a finite‐sample Gaussian model. Using the curvature of the model, we develop new finite‐sample bounds on the distribution of minimum‐distance statistics. These bounds allow us to construct tests for composite hypotheses which are uniformly asymptotically valid over a large class of data generating processes and structural models., Massachusetts Institute of Technology (Castle-Krob Career Development Chair), Alfred P. Sloan Foundation (Sloan Research Fellowship)

This paper examines the issue of weak identification in maximum likelihood, motivated by problems with estimation and inference in a multidimensional dynamic stochastic general equilibrium model. We show that two forms of the classical score (Lagrange multiplier) test for a simple hypothesis concerning the full parameter vector are robust to weak identification. We also suggest a test for a composite hypothesis regarding a subvector of parameters. The suggested subset test is shown to be asymptotically exact when the nuisance parameter is strongly identified. We pay particular attention to the question of how to estimate Fisher information and we make extensive use of martingale theory., Ford Motor Company, National Science Foundation (U.S.) (NSF Graduate Research Fellowship, Grant 1122374), Alfred P. Sloan Foundation (Research Fellowship), Massachusetts Institute of Technology. Department of Economics (Castle-Krob Career Development Chair)

In this paper we connect the discrepancy between two estimates of Fisher information, one based on the quadratic variation of the score and the other based on the negative Hessian of the log-likelihood, to weak identification. Classical asymptotic approximations assume that these two estimates are asymptotically equivalent, but we show that this equivalence fails in many weakly identified models, which can distort the behavior of the MLE. Using a stylized DSGE model we show that the discrepancy between information estimates is large when identification is weak., National Science Foundation (U.S.) (NSF Graduate Research Fellowship, grant no. 1122374), Alfred P. Sloan Foundation (Fellowship), Massachusetts Institute of Technology (Castle-Krob Career Development Chair)

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Economics, 2014., Cataloged from PDF version of thesis., Includes bibliographical references (pages 219-228)., Economic researchers and policymakers need reliable tools both to estimate economic relationships and to measure the uncertainty surrounding their estimates. Unfortunately, economic data sometimes contains limited information useful for estimating relationships of interest. In such cases, the statistical techniques commonly used in applied economics can break down and fail to accurately reflect the level of uncertainty present in the data. If they rely on such tools, researchers and policymakers may come away with serious misconceptions about the precision and reliability of their estimates. Econometricians refer to models where the lack of information in the data causes common statistical techniques to break down as weakly identified. In this thesis, I examine several questions relating to weak identification. In the first chapter, I introduce the class of conditional linear combination tests. These tests control size under weak identification and have a number of optimality properties in a conditional problem. I suggest using minimax regret conditional linear combination tests and propose a computationally tractable class of tests that plug in an estimator for a nuisance parameter. In the second chapter, I consider the problem of detecting weak identification. When weak identification is a concern researchers frequently calculate confidence sets in two steps, first assessing the strength of identification and then deciding whether to use an identification-robust confidence set. Two-step procedures of this sort may generate highly misleading confidence sets, and I demonstrate that two-step confidence sets based on the first stage F-statistic can have extremely poor coverage in linear instrumental variables models with heteroskedastic errors. I introduce a simple approach to detecting weak identification and constructing two-step confidence sets which controls coverage distortions. In the third chapter, joint with Anna Mikusheva, we consider minimum distance statist, by Isaiah Smith Andrews., Ph. D.

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Economics, 2014., Cataloged from PDF version of thesis., Includes bibliographical references (pages 219-228)., Economic researchers and policymakers need reliable tools both to estimate economic relationships and to measure the uncertainty surrounding their estimates. Unfortunately, economic data sometimes contains limited information useful for estimating relationships of interest. In such cases, the statistical techniques commonly used in applied economics can break down and fail to accurately reflect the level of uncertainty present in the data. If they rely on such tools, researchers and policymakers may come away with serious misconceptions about the precision and reliability of their estimates. Econometricians refer to models where the lack of information in the data causes common statistical techniques to break down as weakly identified. In this thesis, I examine several questions relating to weak identification. In the first chapter, I introduce the class of conditional linear combination tests. These tests control size under weak identification and have a number of optimality properties in a conditional problem. I suggest using minimax regret conditional linear combination tests and propose a computationally tractable class of tests that plug in an estimator for a nuisance parameter. In the second chapter, I consider the problem of detecting weak identification. When weak identification is a concern researchers frequently calculate confidence sets in two steps, first assessing the strength of identification and then deciding whether to use an identification-robust confidence set. Two-step procedures of this sort may generate highly misleading confidence sets, and I demonstrate that two-step confidence sets based on the first stage F-statistic can have extremely poor coverage in linear instrumental variables models with heteroskedastic errors. I introduce a simple approach to detecting weak identification and constructing two-step confidence sets which controls coverage distortions. In the third chapter, joint with Anna Mikusheva, we consider minimum distance statist, by Isaiah Smith Andrews., Ph. D.

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Economics, 2014., Cataloged from PDF version of thesis., Includes bibliographical references (pages 219-228)., Economic researchers and policymakers need reliable tools both to estimate economic relationships and to measure the uncertainty surrounding their estimates. Unfortunately, economic data sometimes contains limited information useful for estimating relationships of interest. In such cases, the statistical techniques commonly used in applied economics can break down and fail to accurately reflect the level of uncertainty present in the data. If they rely on such tools, researchers and policymakers may come away with serious misconceptions about the precision and reliability of their estimates. Econometricians refer to models where the lack of information in the data causes common statistical techniques to break down as weakly identified. In this thesis, I examine several questions relating to weak identification. In the first chapter, I introduce the class of conditional linear combination tests. These tests control size under weak identification and have a number of optimality properties in a conditional problem. I suggest using minimax regret conditional linear combination tests and propose a computationally tractable class of tests that plug in an estimator for a nuisance parameter. In the second chapter, I consider the problem of detecting weak identification. When weak identification is a concern researchers frequently calculate confidence sets in two steps, first assessing the strength of identification and then deciding whether to use an identification-robust confidence set. Two-step procedures of this sort may generate highly misleading confidence sets, and I demonstrate that two-step confidence sets based on the first stage F-statistic can have extremely poor coverage in linear instrumental variables models with heteroskedastic errors. I introduce a simple approach to detecting weak identification and constructing two-step confidence sets which controls coverage distortions. In the third chapter, joint with Anna Mikusheva, we consider minimum distance statist, by Isaiah Smith Andrews., Ph. D.

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Economics, 2014., Cataloged from PDF version of thesis., Includes bibliographical references (pages 219-228)., Economic researchers and policymakers need reliable tools both to estimate economic relationships and to measure the uncertainty surrounding their estimates. Unfortunately, economic data sometimes contains limited information useful for estimating relationships of interest. In such cases, the statistical techniques commonly used in applied economics can break down and fail to accurately reflect the level of uncertainty present in the data. If they rely on such tools, researchers and policymakers may come away with serious misconceptions about the precision and reliability of their estimates. Econometricians refer to models where the lack of information in the data causes common statistical techniques to break down as weakly identified. In this thesis, I examine several questions relating to weak identification. In the first chapter, I introduce the class of conditional linear combination tests. These tests control size under weak identification and have a number of optimality properties in a conditional problem. I suggest using minimax regret conditional linear combination tests and propose a computationally tractable class of tests that plug in an estimator for a nuisance parameter. In the second chapter, I consider the problem of detecting weak identification. When weak identification is a concern researchers frequently calculate confidence sets in two steps, first assessing the strength of identification and then deciding whether to use an identification-robust confidence set. Two-step procedures of this sort may generate highly misleading confidence sets, and I demonstrate that two-step confidence sets based on the first stage F-statistic can have extremely poor coverage in linear instrumental variables models with heteroskedastic errors. I introduce a simple approach to detecting weak identification and constructing two-step confidence sets which controls coverage distortions. In the third chapter, joint with Anna Mikusheva, we consider minimum distance statist, by Isaiah Smith Andrews., Ph. D.