Search

Summary A property, or statistical functional, is said to be elicitable if it minimizes the expected loss for some loss function. The study of which properties are elicitable sheds light on the capabilities and limitations of point estimation and empirical risk minimization. While recent work has sought to identify which properties are elicitable, here we investigate a more nuanced question: how many dimensions are required to indirectly elicit a given property? This number is called the elicitation complexity of the property. We lay the foundation for a general theory of elicitation complexity, which includes several basic results on how elicitation complexity behaves and the complexity of standard properties of interest. Building on this foundation, our main result gives tight complexity bounds for the broad class of Bayes risks. We apply these results to several properties of interest, including variance, entropy, norms and several classes of financial risk measures. The article concludes with a discussion and open questions.

SUMMARY A Bayesian adaptive design is proposed for a comparative two-armed clinical trial using decision-theoretic approaches. A loss function is specified, based on the cost for each patient and the costs of making incorrect decisions at the end of a trial. At each interim analysis, the decision to terminate or to continue the trial is based on the expected loss function while concurrently incorporating efficacy, futility and cost. The maximum number of interim analyses is determined adaptively by the observed data. We derive explicit connections between the loss function and the frequentist error rates, so that the desired frequentist properties can be maintained for regulatory settings. The operating character istics of the design can be evaluated on frequentist grounds. Extensive simulations are carried out to compare the proposed design with existing ones. The design is general enough to accommodate both continuous and discrete types of data. We illustrate the methods with an animal study evaluating a medical treatment for cardiac arrest.

SUMMARY We examine large-sample properties of cross-validation for estimating cell probabilities, starting from a completely general measure of loss. Necessary and sufficient conditions on the loss function are derived for the resulting estimator to be consistent, or to minimize expected loss. These results reveal that cross-validation is extremely sensitive to the shape of the loss function. Nevertheless, when the loss function is chosen correctly, cross-validation can be relied on to perform well for large samples. We provide a simple method of generating loss functions with optimal properties. Extension to the estimation of univariate probability density functions is discussed at a heuristic level.

SUMMARY Somerville (1954) proposed a one-stage and a two-stage procedure for the selection of the population with the largest mean from a set of normal populations with unknown means and a common, known variance. The two-stage procedure eliminates one population after the first stage. For these procedures he assumed that a certain loss was incurred when an incorrect selection was made and also that a cost was due to sampling. The sample sizes which would minimize the maximum expected loss, the maximum being taken over all possible configurations of the true population means, were derived. For the special case of three populations he showed that the two-stage procedure, using appropriate allocations of observations between stages, has a smaller maximum expected loss than does the onestage procedure. This paper generalizes Somerville's formulation of the two-stage procedure to an arbitrary finite number of populations and presents results for the special case of four populations, where two two-stage procedures are considered. For the problem of selecting the population with the largest mean from a set of normal populations with unknown means and a common, known variance, Somerville (1954) proposed a one-stage and a two-stage procedure, which eliminates one population after the first stage. He assumed that a certain loss was incurred when an incorrect selection was made and also assumed a cost due to sampling. For these procedures, he derived the sample sizes which minimize the maximum expected loss, the maximum being taken over all possible configurations of the true population means. He showed, for the special case of three populations that the two-stage procedure, using appropriate allocations of observations between stages, has a smaller maximum expected loss than does the one-stage procedure. Other approaches to this problem have been made by Bechhofer, Sobel and Gupta (see Bechhofer, 1954, and Gupta, 1965). In this paper the formulation of Somerville's two-stage procedure is extended to an arbitrary, finite number of populations. For the case of four populations, the expected losses are analysed in detail and numerical results obtained. Two possible two-stage procedures are considered: in the first procedure, one population is discarded and in the second, two populations are discarded after the first stage. It is found that while both procedures, using appropriate allocations, have smaller maximum expected losses than does the corresponding one-stage procedure, which one of the two-stage procedures is the better depends on the allocation. These numerical results required the evaluation of certain multivariate normal integrals with arbitrary correlation matrices and arbitrary limits of integration. This was accomplished for 4- and 5-variate integrals using a method suggested