Search

The question of whether to carry out a quinquennial Census is faced by national statistical offices in increasingly many countries, including Canada, Nigeria, Ireland, Australia, and South Africa. We describe uses and limitations of cost-benefit analysis in this decision problem in the case of the 2016 Census of South Africa. The government of South Africa needed to decide whether to conduct a 2016 Census or to rely on increasingly inaccurate postcensal estimates accounting for births, deaths, and migration since the previous (2011) Census. The cost-benefit analysis compared predicted costs of the 2016 Census to the benefits of improved allocation of intergovernmental revenue, which was considered by the government to be a critical use of the 2016 Census, although not the only important benefit. Without the 2016 Census, allocations would be based on population estimates. Accuracy of the postcensal estimates was estimated from the performance of past estimates, and the hypothetical expected reduction in errors in allocation due to the 2016 Census was estimated. A loss function was introduced to quantify the improvement in allocation. With this evidence, the government was able to decide not to conduct the 2016 Census, but instead to improve data and capacity for producing post-censal estimates.

Summary Latent class models are increasingly used to assess the accuracy of medical diagnostic tests and other classifications when no gold standard is available and the true state is unknown. When the latent class is treated as the true class, the latent class models provide measures of components of accuracy including specificity and sensitivity and their complements, type I and type II error rates. The error rates according to the latent class model differ from the true error rates, however, and empirical comparisons with a gold standard suggest the true error rates often are larger. We investigate conditions under which the true type I and type II error rates are larger than those provided by the latent class models. Results from Uebersax (1988, Psychological Bulletin 104, 405–416) are extended to accommodate random effects and covariates affecting the responses. The results are important for interpreting the results of latent class analyses. An error decomposition is presented that incorporates an error component from invalidity of the latent class model.

Average accuracy of jury verdicts for a set of cases can be studied empirically and systematically even when the correct verdict cannot be known. The key is to obtain a second rating of the verdict, for example, the judge’s, as in the recent study of criminal cases in the United States by the National Center for State Courts (NCSC). That study, like the famous Kalven-Zeisel study, showed only modest judge-jury agreement. Simple estimates of jury accuracy can be developed from the judge-jury agreement rate; the judge’s verdict is not taken as the gold standard. Although the estimates of accuracy are subject to error, under plausible conditions they tend to overestimate the average accuracy of jury verdicts. The jury verdict was estimated to be accurate in no more than 87 percent of the NCSC cases (which, however, should not be regarded as a representative sample with respect to jury accuracy). More refined estimates, including false conviction and false acquittal rates, are developed with models using stronger assumptions. For example, the conditional probability that the jury incorrectly convicts given that the defendant truly was not guilty (a “Type I error”) was estimated at 0.25, with an estimated standard error (s.e.) of 0.07, the conditional probability that a jury incorrectly acquits given that the defendant truly was guilty (“Type II error”) was estimated at 0.14 (s.e. 0.03), and the difference was estimated at 0.12 (s.e. 0.08). The estimated number of defendants in the NCSC cases who truly are not guilty but are convicted does seem to be smaller than the number who truly are guilty but are acquitted. The conditional probability of a wrongful conviction, given that the defendant was convicted, is estimated at 0.10 (s.e. 0.03).

We explore whether less detailed probabilistic hazard maps might perform better by assessing how smoothing Japan's national earthquake hazard maps affects their fit to a 510-year record of shaking. As measured by the fractional exceedance metric implicit in such probabilistic hazard maps, simple smoothing over progressively larger areas improves the maps' performance such that in the limit a uniform map performs best. However, using the squared misfit between maximum observed shaking and that predicted as a metric, map performance improves up to a ~75-150 km smoothing window, and then decreases with further smoothing. This result suggests that the probabilistic hazard models and the resulting maps may be over-parameterized, in that including too high a level of detail to describe past and future earthquakes may lower the maps' ability to predict future shaking.

The sensitivity of a benefit-cost analysis of a data program to incorrect specifications of the cost function C and benefit function B is an important consideration for producers and consumers of the analyses. We consider the effects of misspecification of B or C on the optimal data expenditure and optimal data quality. Geometric and analytical results show when the analysis will be sensitive and when order of magnitude estimates for costs and benefits will suffice in practice. A benefit-cost analysis of the 1970 census illustrates these results. We also show why misspecification of benefits typically has a larger effect than comparable misspecification of costs. If the benefits and the costs are both large but their difference small, the optimal expenditure for data (or optimal data quality) will be highly sensitive to changes in specifications of costs, benefits, data use, etc. In other cases, the sensitivity of the prescribed optimal expenditure to scalemisspecification of costs or benefits depends on the sharpness of the bend of the graph of B versus C near the optimum. Although the prescribed optimal expenditure is the same when B is misspecified as when C is comparably misspecified, the levels of data quality attained can differ. When B is misspecified, the data quality attained decreases if benefits are understated. When C is misspecified, the data quality attained usually decreases if costs are overstated, but for some unusual decision problems it increases and in rare, ‘self-correcting’ problems, it remains unchanged.

In July 1991 the Census Bureau recommended to its parent agency, the Department of Commerce, that the 1990 census be adjusted for undercount. The Secretary of Commerce decided not to adjust, however. Those decisions relied at least partly on the Census Bureau's analyses of the accuracy of the census and of the proposed undercount adjustments based on the Post-Enumeration Survey (PES). Error distributions for the nation, states, and smaller geographic units were estimated with extensions of methods applied to test censuses. To summarize and assess the relative importance of errors in different units, the Census Bureau used aggregate loss functions. This article describes the total error analysis and loss function analysis of the Census Bureau. In its decision not to adjust the census, the Department of Commerce cited different criteria than aggregate loss functions. Those criteria are identified and discussed.

This article considers surveys where one can observe, after the sample is selected, that each member of the sample belongs to one or more aggregations. The population of aggregations is of interest, and we consider the probability that a given aggregation contains at least one sample member. An expression for the probability is derived as a function of population parameters, many of which can be known only if additional, costly data collection is undertaken. A variety of model-based estimators of those parameters is discussed, and their relative advantages and disadvantages are noted. The estimators are applied to a particular case, the National Educational Longitudinal Study of 1988 (NELS:88), in which the aggregations are tenth-grade schools. Evaluations of the estimators are presented. In NELS:88, a sample of eighth-grade schools and eighth-grade students in those schools was surveyed and 2 years later, when most of the students were in tenth grade, the students were resurveyed. If the tenth-grade school sample (i.e., the set of tenth-grade schools enrolling one or more sampled students) is a probabiliy sample, then we can make inferences to the population of tenth-grade schools. A requirement for a sample to be a probability sample is that the selection probabilities be nonzero for all units and be known for the selected units. This article discusses how those probabilities can be estimated from complete and incomplete data. Various estimators based on incomplete data are derived and empirically evaluated with data from a special test sample of schools and data from the Houston Independent School District.

The amount worth spending to collect and analyze data depends on the uses of the data. We consider the determination of the optimal expenditure in some simple situations when only some aspects of the data use are known. In particular, we consider uses where an action is taken if a statistic exceeds a threshold τ, which is unknown at the time of data planning and collection (though its value will not depend on the value of the statistic). We analyze the sensitivity of the optimal expenditure to the a priori uncertainty in τ. For many decision problems, the optimal expenditure decreases with increasing uncertainty about τ, but we identify some uses for which the optimal expenditure increases with increasing uncertainty about τ. We also study the effects of other forms of ambiguity. Uncertainty about whether the data will be used typically decreases the optimal expenditure, as does uncertainty about the preferences represented in the decision making.

Official forecasts of mortality depend on assumptions about target values for the future rates of decline in mortality rates. Smooth functions connect the jump‐off (base‐year) mortality to the level implied by the targets. Three alternative sets of targets are assumed, leading to high, middle, and low forecasts. We show that this process can be closely modeled using simple linear statistical models. These explicit models allow us to analyze the error structure of the forecasts. We show that the current assumption of perfect correlation between errors in different ages, at different forecast years, and for different causes of death, is erroneous. An alternative correlation structure is suggested, and we show how its parameters can be estimated from the past data. The effect of the level of aggregation on the accuracy of mortality forecasts is considered. It is not clear whether or not age‐ and cause‐specific analyses have been more accurate in the past than analyses based on age‐specific mortality alone wo...

"We describe a methodology for estimating the accuracy of dual systems estimates (DSE's) of population, census estimates of population, and estimates of undercount in the census. The DSE's are based on the census and a post-enumeration survey (PES). We apply the methodology to the 1988 dress rehearsal census of St. Louis and east-central Missouri and we discuss its applicability to the 1990 [U.S.] census and PES. The methodology is based on decompositions of the total (or net) error into components, such as sampling error, matching error, and other nonsampling errors. Limited information about the accuracy of certain components of error, notably failure of assumptions in the 'capture-recapture' model, but others as well, lead us to offer tentative estimates of the errors of the census, DSE, and undercount estimates for 1988. Improved estimates are anticipated for 1990." Comments are included by Eugene P. Ericksen and Joseph B. Kadane (pp. 855-7) and Kenneth W. Wachter and Terence P. Speed (pp. 858-61), as well as a rejoinder by Mulry and Spencer (pp. 861-3).

The Office of the Actuary, U.S. Social Security Administration, produces alternative forecasts of mortality to reflect uncertainty about the future. Appropriate probabilistic interpretations of the intervals have not been provided, however, because explicit stochastic models are not used. In this article we identify the components and assumptions of the official forecasts and approximate them by stochastic parametric models. We estimate parameters of the models from past data, derive statistical intervals for the forecasts, and compare them with the official high-low intervals. We use the models to evaluate the forecasts rather than to develop different predictions of the future. Analysis of data from 1972 to 1985 shows that the official intervals for mortality forecasts for males or females aged 45–70 have approximately a 95% chance of including the true mortality rate in any year. For other ages the chances are much less than 95%. The Office of the Actuary's interval estimates of age- and cause...

PART I: USES OF STATISTICS FOR DESCRIPTION PART II DATA COLLECTION PART III USES OF STATISTICS FOR POLICY ANALYSIS

Two problems that arise in apportioning the U.S. House of Representatives are: (a) fractional numbers of representatives cannot be allocated, so states receive different per capita representation, and (b) the state population sizes are known only with error. Both problems are addressed in a unified way with decision theory. Although the method currently in use, equal proportions, has poor properties when the populations are assumed perfectly known, it performs surprisingly well in the presence of modest errors in the data. The converse is true for the quota method. Previously developed qualitative notions of bias in apportionment methods are extended to provide a quantitative definition of bias. The new definition accounts both for bias in the apportionment method and for bias arising from imperfect population measurements. Illustrative estimates of the bias against states with large black populations are developed.

Educational and psychological tests traditionally have informed judgments concerning individuals, but recently they have been widely used for comparing groups. Such uses include program evaluations (e.g., to what extent does Title I improve achievement), analysis of test score trends (e.g., the SAT decline) (Harnischfeger & Wiley, 1975), and comparison of contemporary subpopulations (e.g., achievement levels of black and white youth) (Burton & Jones, 1982). Using test scores (or item responses) to make group comparisons involves summarizing individuals' characteristics into a measure for the group. In many if not most applications, the measure chosen has been the average (i.e., arithmetic mean). But averaging scores (ability levels) is invalid if the scores (or abilities) are ordinal rather than cardinal attributes. In particular, group comparisons can change completely under order-preserving transformations of the test scales. This is illustrated by real and artificial examples. The notion of stochastic ordering is applied to develop group comparisons that are invariant under order-preserving transformations of the test scale.

"Errors in population forecasts arise from errors in the jump-off population and errors in the predictions of future vital rates. The propagation of these errors through the linear (Leslie) growth model is studied, and prediction intervals for future population are developed. For U.S. national forecasts, the prediction intervals are compared with the U.S. Census Bureau's high-low intervals." In order to assess the accuracy of the predictions of vital rates, the authors "derive the predictions from a parametric statistical model and estimate the extent of model misspecification and errors in parameter estimates. Subjective, expert opinion, so important in real forecasting, is incorporated with the technique of mixed estimation. A robust regression model is used to assess the effects of model misspecification."

This paper presents at an elementary level a unified presentation of concepts related to sufficiency and minimal sufficiency. Extensively discussed are techniques for showing in a particular statistical model that a given statistic is not sufficient or that a given sufficient statistic is not minimal. The applicability of these techniques is illustrated in three examples.

If test scores are considered ordinal but not cardinal attributes, the average test score may not be a meaningful way to summarize the scores of a group of individuals. Similarly, correlation coefficients may not form meaningful summary measures of association between test scores. Using the theory of stochastic ordering and quadrant dependence, new techniques that do not depend on any particular test scale are developed for reporting, interpreting, and comparing distributions of test scores. The use of a relevant test scale is considered also, as well as measures of distance between distributions.