Search

BackgroundIn the rapidly changing industrial environment and job market, engineering profession requires a vast body of skills, one of them being decision making under uncertainty. Knowing that misunderstanding of probability concepts can lead to wrong decisions, the main objective of this study is to investigate the presence of probability misconceptions among undergraduate students of electrical engineering. Five misconceptions were investigated:insensitivity to sample size,base rate neglected,misconception of chance,illusory correlation, andbiases in the evaluation of conjunctive and disjunctive events.The study was conducted with 587 students who attended bachelor schools of electrical engineering at two universities in Serbia. The presence of misconceptions was tested using multiple-choice tasks. This study also introduces a novel perspective, which is reflected in examination of the correlation between students’ explanations of given answers and their test scores.ResultsThe results of this study show that electrical engineering students are, susceptible to misconceptions in probability reasoning. Although future engineers from the sample population were most successful in avoiding misconceptions of chance, only 35% of examinees were able to provide a meaningful explanation. Analysis of students’ explanations, revealed that in many cases majority of students were prone to common misconceptions. Among the sample population, significant percentage of students were unable to justify their own answers even when they selected the correct option. The results also indicate that formal education in probability and statistics did not significantly influence the test score.ConclusionsResults of the present study indicate a need for further development of students’ deep understanding of probability concepts, as well as the need for the development of competencies that enable students to validate their answers. The study emphasizes the importance of answer explanations, since they allow us to discover whether students who mark the correct answer have some misconceptions or may be prone to some other kind of error. We found that the examinees who failed to explain their choices had much lower test scores than those who provided some explanation.

Background and aims: To estimate sample size for Cohen’s kappa agreement test can be challenging especially when it is expected that the true marginal rating frequencies are not the same. This study aims to present the tables that display a minimum sample size determination for an agreement test when certain assumptions are hold. Method: We adopted the sample size formula provided by Flack and colleagues (1988) to calculate the minimum sample sizes required using PASS software. The power is pre-specified to be at least 80% and the alpha to be less than 0.05. The effect sizes were derived from several pre-specified estimates such as the pattern of the true marginal rating frequencies and the difference between the two kappa coefficients in the hypothesis testing. Results: When the true marginal rating frequencies are the same, the minimum sample size determination can range from 2 to 698 depending on the actual value of the effect size. When the true marginal rating frequencies are not the same, then the majority of the minimum sample size required for this condition is more than double than that required sample size when the true marginal rating frequencies are the same. Conclusion: Concerning that the sample size formula could produce a very extreme small sample size, therefore, the determination of K1 and K2 should be based on reasonable estimates. We recommend for all sample size determinations for Cohen’s kappa agreement test, the true marginal rating frequencies can be assumed the same. Otherwise, it will be necessary to multiple the estimated minimum sample size by two to accommodate if the true marginal rating frequencies is not the same.

Purpose Operations managers are subjected to various cognitive biases, which may lead them to make less optimal decisions as suggested by the normative models. In their seminal work, Tversky and Kahneman introduced three heuristics based on which people make decisions: representativeness, availability, and anchoring. This paper aims to investigate the six cognitive biases resulting from the use of the representativeness heuristic, namely, insensitivity to prior probability of outcomes, insensitivity to sample size, misconception of chance, insensitivity to predictability, the illusion of validity, and misconception of regression. Specifically, the paper examines how cognitive reflection and training affect these six cognitive biases in the operations management context. Methods For each cognitive bias, a scenario related to operations management was developed. The participants of the experimental study are asked to select among three responses, where one response is correct and the other two are biased. A total of 315 students from the University of North Texas participated in this study and 302 valid responses were used in the analysis. Results The results show that in all six scenarios, >50% of the respondents make biased decisions. However, using simple training, the bias is significantly reduced. Regarding the relationship between cognitive biases and cognitive reflection, the results partially support the hypothesis that people with high cognitive reflection ability tend to make less biased decisions. Regarding the effect of training on making biased decisions, the results show that making people aware of the existence of cognitive biases helps them partially to avoid making biased decisions. Conclusion Overall, our study demonstrates the value of training in helping operations managers make less biased decisions. Our discussion section offers some related guidelines for creating a professional environment where the effect of the representativeness heuristic is minimized.

Purpose Heuristics are used in the judgment and decision-making process of bank employees; however, discussions and research on the type or range of judgmental heuristics are very difficult to find throughout the world. In light of this, the purpose of this paper is to empirically analyze what types of heuristics are used in bank employees’ judgment and decision-making processes and the extent to which those types of heuristics prevent rational decision making due to the systematic biases they generate. In particular, this study aims to conduct empirical research based on various scenarios related to the banking industry. Design/methodology/approach To examine the heuristics in decision-making circumstances and the level of subsequent biases, the present study narrowed the scope of research to the three main types of heuristics introduced by Tversky and Kahneman (1974), namely, representativeness heuristics, availability heuristics and anchoring and adjustment heuristics. To analyze the bank employees’ decision making, this study specifically investigated the level of decision-making heuristics and the level of bias by focusing on these three types of heuristics. This study targeted bank employees who either sell financial products or are engaged in customer service work at a real/physical bank. Findings For representativeness heuristics, this study found bank employees’ judgment of probability was influenced by biases, such as insensitivity to prior probability, insensitivity to sample size, misconception of chance and insensitivity to predictability. Regarding availability heuristics, it found that bank employees judge the probability of events based on the ease of recalling an event instead of the actual frequency of the event, and so they fall prey to systematic biases. Finally, regarding anchoring and adjustment heuristics, this study found that employees fall prey to judgment biases as they judge the probability of conjunctive events and disjunctive events based on anchoring and insufficient adjustment. Originality/value Although people who are well-trained in statistics can avoid rudimentary errors, they fall prey to biased judgment at a similar level to those who are not properly trained in statistics when it comes to more complicated and ambiguous issues. It clearly indicates that it is risky to determine that financial experts would be more rational than the general public in making various judgments required in the policy-making process. To conclude, it is imperative to recognize the existence of heuristics-based systematic biases in the judgment and decision-making process and, furthermore, to reinforce the education and training system to improve bank employees’ rational choice and judgment ability.

Success in the physical and social worlds often requires knowledge of population size. However, many populations cannot be observed in their entirety, making direct assessment of their size difficult, if not impossible. Nevertheless, an unobservable population size can be inferred from observable samples. We measured people's ability to make such inferences and their confidence in these inferences. Contrary to past work suggesting insensitivity to sample size and failures in statistical reasoning, inferences of populations size were accurate-but only when observable samples indicated a large underlying population. When observable samples indicated a small underlying population, inferences were systematically biased. This error, which cannot be attributed to a heuristics account, was compounded by a metacognitive failure: Confidence was highest when accuracy was at its worst. This dissociation between accuracy and confidence was confirmed by a manipulation that shifted the magnitude and variability of people's inferences without impacting their confidence. Together, these results (a) highlight the mental acuity and limits of a fundamental human judgment and (b) demonstrate an inverse relationship between cognition and metacognition.

The sample size calculation plays an important role in many areas, particularly for applications in the biomedical and social sciences. Large-sample size is wasting of time and resources. S...

The main objective of sampling is to obtain a representative sample for an unbiased and efficient estimate within a budget constraint. In a balanced sample, according to Yates’ definition (Yates, 1971), the mean value of the balanced factor in the sample is equal to the mean of the factor in the population. In this study, a balanced sample is not a purposively selected sample but a randomly selected one. Another important reason for a balanced sample is to protect the inference against a model misspecification (Royall & Herson, 1973a; Royall & Herson, 1973b). In this work, we propose and demonstrate a practical balancing method which would be a small modification to currently practiced design-based sampling procedures for small and large-scale surveys. We demonstrate practicality of our approach with a simulation of sample selection from 3,143 U.S. Counties for estimates of the total and mean population sizes in 2010 with Census 2000 count and State indicator as auxiliary variables. Our simulation study indicated that a balanced sample was good for reducing bias regardless of the particular sorting method. Rather than selecting a random sample from an ordered frame, we should try to find a balanced sample for an unbiased estimate.

Determining the appropriate sample size for a study, whatever be its type, is a fundamental aspect of biomedical research. An adequate sample ensures that the study will yield reliable information, regardless of whether the data ultimately suggests a clinically important difference between the interventions or elements being studied. The probability of Type 1 and Type 2 errors, the expected variance in the sample and the effect size are the essential determinants of sample size in interventional studies. Any method for deriving a conclusion from experimental data carries with it some risk of drawing a false conclusion. Two types of false conclusion may occur, called Type 1 and Type 2 errors, whose probabilities are denoted by the symbols σ and β. A Type 1 error occurs when one concludes that a difference exists between the groups being compared when, in reality, it does not. This is akin to a false positive result. A Type 2 error occurs when one concludes that difference does not exist when, in reality, a difference does exist, and it is equal to or larger than the effect size defined by the alternative to the null hypothesis. This may be viewed as a false negative result. When considering the risk of Type 2 error, it is more intuitive to think in terms of power of the study or (1 − β). Power denotes the probability of detecting a difference when a difference does exist between the groups being compared. Smaller α or larger power will increase sample size. Conventional acceptable values for power and α are 80% or above and 5% or below, respectively, when calculating sample size. Increasing variance in the sample tends to increase the sample size required to achieve a given power level. The effect size is the smallest clinically important difference that is sought to be detected and, rather than statistical convention, is a matter of past experience and clinical judgment. Larger samples are required if smaller differences are to be detected. Although the principles are long known, historically, sample size determination has been difficult, because of relatively complex mathematical considerations and numerous different formulas. However, of late, there has been remarkable improvement in the availability, capability, and user-friendliness of power and sample size determination software. Many can execute routines for determination of sample size and power for a wide variety of research designs and statistical tests. With the drudgery of mathematical calculation gone, researchers must now concentrate on determining appropriate sample size and achieving these targets, so that study conclusions can be accepted as meaningful.

Sample size is very important in statistical research because it is not too small or too large. Given significant level α, the sample size is calculated based on the z-value and pre-defined error. Such error is defined based on the previous experiment or other study or it can be determined subjectively by specialist, which may cause incorrect estimation. Therefore, this research proposes an objective method to estimate the sample size without pre-defining the error. Given an available sample X = {X1, X2, ..., Xn}, the error is calculated via the iterative process in which sample X is re-sampled many times. Moreover, after the sample size is estimated completely, it can be used to collect a new sample in order to estimate new sample size and so on.

Sample size estimation is very crucial in any research design. A research design with less sample size may give a biased result or inconclusive result. A research design with very large sample size than required results is waste of resources, time and energy. So, it is very essential to determine ‘ideal’ or ‘optimum’ sample size. This article gives formulae and R code for determining sample size for single mean, two means, single proportion, two proportions, proportion in survey type data, case control studies, cohort studies, correlation coefficient and difference between correlation coefficients.

Asking advice about sample size calculation is one of frequent requests from clinical researchers to statisticians. Sample size calculation is essential to obtain the results as the researcher expects as well as to interpret the statistical results as reasonable one. Usually insignificant results from studies with too small sample size may be subjects to suspicion about false negative, and also significant ones from those with too large sample size may be subjects to suspicion about false positive. In this article, the principles in sample size calculation will be introduced and practically some examples will be displayed using a free software, G*Power.1

This chapter will show you how random sampling ensures a representative sample (on average) and makes it possible for you to describe (approximately) how different your results (from the sample) are from the (unknown) characteristics of the population. By using a random sample, your survey results will be approximately correct (compared with what you would find by interviewing all customers), and you will know if your results are close enough for comfort. The language we use when sampling helps us distinguish the population (that we wish to know about) from the sample (that we have selected to observe). A population is any collection of units that you are interested in knowing about. A sample is a smaller collection of units selected from the population. A sample is representative if characteristics arise with the same percentages in the sample as in the population. A biased sample is not representative in an important way. A sampling frame gives you access to the population units so that a random sample can be chosen. A sample statistic is any number computed from your sample data. A population parameter is any number computed for the entire population. An estimator is a description of a sample statistic used as a guess for the value of a population parameter, and the actual number computed from the data is called an estimate. The error of estimation is the estimator (or estimate) minus the population parameter and is usually unknown. An unbiased estimator is correct on average (neither systematically too high nor too low). A pilot study is a small-scale version of a study, designed to help you identify problems and fix them before the real study is run. A random sample consists of independently chosen units where each population unit has equal probability of being chosen. The central limit theorem is an amazing mathematical fact about random sampling that tells you that the average of the sample values follows a distribution that becomes more normal-shaped as the sample size grows, tells you that the mean of the sample average is the population mean, and tells you that the standard deviation of the sample average (which indicates the quality of the sample information) is the population standard deviation divided by the square root of the sample size. The standard error of a sample statistic indicates approximately how far the statistic is from its population value. The standard error of the average tells approximately how far the sample average is from the unknown population mean.

Minimum Bayes factors are commonly used to transform two-sided p-values to lower bounds on the posterior probability of the null hypothesis. Several proposals exist in the literature, but none of them depends on the sample size. However, the evidence of a p-value against a point null hypothesis is known to depend on the sample size. In this article, we consider p-values in the linear model and propose new minimum Bayes factors that depend on sample size and converge to existing bounds as the sample size goes to infinity. It turns out that the maximal evidence of an exact two-sided p-value increases with decreasing sample size. The effect of adjusting minimum Bayes factors for sample size is shown in two applications.

The vast majority of settings for which frequentist statistical properties are derived assume a fixed, a priori known sample size. Familiar properties then follow, such as, for example, the consistency, asymptotic normality, and efficiency of the sample average for the mean parameter, under a wide range of conditions. We are concerned here with the alternative situation in which the sample size is itself a random variable which may depend on the data being collected. Further, the rule governing this may be deterministic or probabilistic. There are many important practical examples of such settings, including missing data, sequential trials, and informative cluster size. It is well known that special issues can arise when evaluating the properties of statistical procedures under such sampling schemes, and much has been written about specific areas (Grambsch P. Sequential sampling based on the observed Fisher information to guarantee the accuracy of the maximum likelihood estimator. Ann Stat 1983; 11: 68–77; Barndorff-Nielsen O and Cox DR. The effect of sampling rules on likelihood statistics. Int Stat Rev 1984; 52: 309–326). Our aim is to place these various related examples into a single framework derived from the joint modeling of the outcomes and sampling process and so derive generic results that in turn provide insight, and in some cases practical consequences, for different settings. It is shown that, even in the simplest case of estimating a mean, some of the results appear counterintuitive. In many examples, the sample average may exhibit small sample bias and, even when it is unbiased, may not be optimal. Indeed, there may be no minimum variance unbiased estimator for the mean. Such results follow directly from key attributes such as non-ancillarity of the sample size and incompleteness of the minimal sufficient statistic of the sample size and sample sum. Although our results have direct and obvious implications for estimation following group sequential trials, there are also ramifications for a range of other settings, such as random cluster sizes, censored time-to-event data, and the joint modeling of longitudinal and time-to-event data. Here, we use the simplest group sequential setting to develop and explicate the main results. Some implications for random sample sizes and missing data are also considered. Consequences for other related settings will be considered elsewhere. IAP research Network of the Belgian Government (Belgian Science Policy) - NIH

Catch-at-age (or catch-at-length) data are one of the major components of most modern statistical stock assessment methods. Catch-at-age data provide, among other things, information about gear selectivity and recruitment strength. Catch-at-age data can also have a large influence on the estimates of fishing mortality, absolute abundance, and trends in abundance. The multinomial distribution describes the theoretical sampling process that is used to collect catch-at-age data, but only under the assumption of random sampling. Sampling designs generally employed to collect fishery-related data lead to age–composition estimates that depart from the strict theoretical multinomial probability distribution. Lack of independence can, for example, be due to size- or age-specific schooling or aggregating, causing positive correlations among individuals, and overdispersion. An additional cause of inadequacy of the multinomial assumption is model misspecification. Therefore, the effective sample size that should be used in an assessment model can be much smaller than the actual sample size. This can cause inappropriate weighting among data sets and negatively biased estimates of uncertainty. I use simulation analysis to evaluate five methods to estimate the effective sample size for catch-at-age data: (1) iterative multinomial likelihood; (2) normal approximation, using binomial variance; (3) lognormal likelihood with variance proportional to the inverse of the proportion; (4) Dirichlet likelihood; and (5) a multivariate normal approximation. The results show that all five methods perform similarly, but do not reduce estimation error relative to using the actual sample size unless the effective sample size is about one fifth of the actual sample size. All but (4) produced positively biased estimates of the effective sample size. If the effective sample size is not known within half an order of magnitude, I recommend using the lognormal likelihood with variance proportional to the inverse of the proportion and a regression against the actual sample size. This method is less computationally intense than (1), more robust than (2) and (4), and produces the least biased estimates of effective sample size, except for (4). Unlike (1), it can be included in Bayesian analysis.

Finding an "appropriate sample size" has been the most basic and foremost problem; a research worker is always faced with, in all sampling based analytical researches. This is so, since a very large sized sample results to unnecessary wastage of resources, while a very small sized sample may affect adversely the accuracy of sample estimates and thus in turn losing the very efficacy of selected sampling plan. The present paper attempts to highlight the main determinant factors and the analytical approach towards estimation ofrequired sample size, along with a few illustrations. DOI: http://dx.doi.org/10.3126/jucms.v2i1.10493 Journal of Universal College of Medical Sciences (2014) Vol.2(1): 45-47

This article provides a way to determine adequate sample size for the confidence interval of covariate-adjusted mean difference in randomized experiments. The standard error of adjusted mean difference depends on covariate variance and balance, which are two unknown quantities at the stage of planning sample size. If covariate observations are viewed as randomly varying from one study to another, the covariate variance and balance are related to a t statistic in the standard error of adjusted mean difference. Using this t statistic in the standard error, one can express the expected width of the confidence interval as a function of the sample size. Alternatively, a sample size can be found to achieve a desired probability of having the width of the confidence interval smaller than a predetermined upper bound.

In this paper, we establish some Pitman closeness results concerning the sample median from a symmetric continuous distribution. We show that when an odd sample size is increased by one, the sample median becomes Pitman-closer to the population median, while when an even sample size is increased by one, the sample median need not be Pitman-closer. We establish the former through probabilistic derivations while the latter is through a counterexample. We also discuss the situation when the sample is increased by two observations.

The present research concerns the hypothesis that intuitive estimates of the arithmetic mean of a sample of numbers tend to increase as a function of the sample size; that is, they reflect a systematic sample size bias. A similar bias has been observed when people judge the average member of a group of people on an inferred quantity (e.g., a disease risk; see Price, 2001; Price, Smith, & Lench, 2006). Until now, however, it has been unclear whether it would be observed when the stimuli were numbers, in which case the quantity need not be inferred, and “average” can be precisely defined as the arithmetic mean. In two experiments, participants estimated the arithmetic mean of 12 samples of numbers. In the first experiment, samples of from 5 to 20 numbers were presented simultaneously and participants quickly estimated their mean. In the second experiment, the numbers in each sample were presented sequentially. The results of both experiments confirmed the existence of a systematic sample size bias.

The sample size determination is an important issue for planning research. However, limitations in size have seldom been discussed in the literature. Thus, how to allocate participants into different treatment groups to achieve the desired power is a practical issue that still needs to be addressed when one group size is fixed. The authors focused on the sample size determination for the second group for the two-sample K. Yuen's (1974) trimmed mean test in the condition of heterogeneous variances. The results show that the sample size needed is less than that of the traditional B.L. Welch's test (1938), especially for nonnormal distributions. Simulation results also demonstrate the accuracy of the proposed formula in terms of Type I error and statistical power.

In this study, we explore the relationship among sample size, variability of a variable, power of the statistical test and effect size in complex cross-sectional studies. The argument that power analysis can be used to optimize the resource usage through determining the smallest sample size needed for a study to assess the detectable effect size has been found questionable. It is worse when dealing with independent variables, which have more than two groups to be compared because the null hypothesis and alternative hypothesis keep on changing with the sample size. Any population mean difference, so long as it is absolutely not equal to zero, can be found statistically significant depending on the sample size, holding other factors constant. The study suggests that, in complex cross-sectional studies, it is very difficult, if not impossible, to determine the appropriate sample size for a hypothesis test in the planning stage of a study.

In clinical trials with t-distributed test statistics the required sample size depends on the unknown variance. Taking estimates from previous studies often leads to a misspecification of the true value of the variance. Hence, re-estimation of the variance based on the collected data and re-calculation of the required sample size is attractive. We present a flexible method for extensions of fixed sample or group-sequential trials with t-distributed test statistics. The method can be applied at any time during the course of the trial and does not require the necessity to pre-specify a sample size re-calculation rule. All available information can be used to determine the new sample size. The advantage of our method when compared with other adaptive methods is maintenance of the efficient t-test design when no extensions are actually made. We show that the type I error rate is preserved.

The aim of this study was to initiate the exploration of debiasing methods applicable in real-life settings for achieving lasting improvement in decision-making competence regarding multiple decision biases. Here, we tested the potentials of the analogical encoding method for decision debiasing. The advantage of this method is that it can foster the transfer from learning abstract principles to improving behavioral performance. For the purpose of the study, we devised an analogical debiasing technique for ten biases (covariation detection, insensitivity to sample size, base rate neglect, regression to the mean, outcome bias, sunk cost fallacy, framing effect, anchoring bias, overconfidence bias, planning fallacy) and assessed the susceptibility of the participants (N = 154) to these biases before and four weeks after the training. We also compared the effect of the analogical training to the effect of an ‘awareness training’ and a ‘no-training’ control group. Results suggested improved performance of the analogical training group only on tasks where the violations of statistical principles are measured. The interpretation of these findings require further investigation, yet it is possible that analogical training may be the most effective in the case of learning abstract concepts, such as statistical principles, which are otherwise difficult to master. The study encourages a systematic research of debiasing trainings and the development of intervention assessment methods to measure the endurance of behavior change in decision debiasing.

We consider sample size calculations for testing differences in means between two samples and allowing for different variances in the two groups. Typically, the power functions depend on the sample size and a set of parameters assumed known, and the sample size needed to obtain a prespecified power is calculated. Here, we account for two sources of variability: we allow the sample size in the power function to be a stochastic variable, and we consider estimating the parameters from preliminary data. An example of the first source of variability is nonadherence (noncompliance). We assume that the proportion of subjects who will adhere to their treatment regimen is not known before the study, but that the proportion is a stochastic variable with a known distribution. Under this assumption, we develop simple closed form sample size calculations based on asymptotic normality. The second source of variability is in parameter estimates that are estimated from prior data. For example, we account for variability in estimating the variance of the normal response from existing data which are assumed to have the same variance as the study for which we are calculating the sample size. We show that we can account for the variability of the variance estimate by simply using a slightly larger nominal power in the usual sample size calculation, which we call the calibrated power. We show that the calculation of the calibrated power depends only on the sample size of the existing data, and we give a table of calibrated power by sample size. Further, we consider the calculation of the sample size in the rarer situation where we account for the variability in estimating the standardized effect size from some existing data. This latter situation, as well as several of the previous ones, is motivated by sample size calculations for a Phase II trial of a malaria vaccine candidate.

This article deals with sample size (n) estimation by computer simulation using the variance of consumer and in-house sensory tests. The current sample size used in descriptive analysis was also examined. The sample size ranging from 20 to 200 was used to detect a difference ranging from 0.0 to 1.0 on a 9-point hedonic scale. For the simulated consumer data, significant difference was first observed at a difference of 0.60 when n = 40. Increasing the difference to above 0.60, the first significance likewise appeared when n = 40. Based on the variability used in the study, these results support the commonly cited sample size of 40–100. Thus, the role of the difference to be detected is very important in determining the sample size (base size). The variance can be obtained from historical data. In this study, the difference ranging from 0.60 to 1.00 is critical and can be used by the research and sensory analysts as a guide. Therefore, the analysts must have an idea of the size of hedonic differences among formula or products to be compared in determining the appropriate sample size. The power of the test for various sample sizes is provided through plots of power versus n. For descriptive analysis, the results showed that the recommended minimum is n = 5, noting that in most cases, judges differed by 1 unit on a 15-point rating scale. The power of the test is also reported for n ranging from 5 to 14 judges.

A simple statistic is proposed for testing the complete independence of random variables having a multivariate normal distribution. The asymptotic null distribution of this statistic, as both the sample size and the number of variables go to infinity, is shown to be normal. Consequently, this test can be used when the number of variables is not small relative to the sample size and, in particular, even when the number of variables exceeds the sample size. The finite sample size performance of the normal approximation is evaluated in a simulation study and the results are compared to those of the likelihood ratio test.

Important similarities and differences are known to exist between the von Mises and wrapped normal distributions, two of the principal models for circular data. In this paper, we consider likelihood-based approaches to determining the sample size required in order to reliably discriminate between the two models. We make use of three new misclassification probability-based criteria to establish lower and upper bounds for the sample size.

In a survey of journal articles, test manuals, and test critique books, the author found that a mean sample size (N) of 260 participants had been used for reliability studies on 742 tests. The distribution was skewed because the median sample size for the total sample was only 90. The median sample sizes for the internal consistency, retest, and interjudge reliabilities were 182, 64, and 36, respectively. The author presented sample size statistics for the various internal consistency methods and types of tests. In general, the author found that the sample sizes that were used in the internal consistency studies were too small to produce sufficiently precise reliability coefficients, which in turn could cause imprecise estimates of examinee true-score confidence intervals. The results also suggest that larger sample sizes have been used in the last decade compared with those that were used in earlier decades.

Deciding upon the optimal sample size in advance is a difficult problem in general. Often, the investigator regrets not having drawn a larger sample; in many cases additional observations are done. This implies that the actual sample size is no longer deterministic; hence, even if all sample elements are drawn at random, the final sample is not a simple random sample. Although this fact is widely recognized, its consequences are often grossly underrated in our view. Too often, these consequences are ignored: the usual statistical procedures are still applied. This paper shows in detail the dangers of applying standard techniques to extended samples. To allow theoretical derivations only some elementary situations are considered. More precisely, the following features hold throughout the paper: - the population variable of interest is normally distributed; - estimation concerns population mean and variance; - all sample elements are drawn at random, with replacement; - only standard estimators, like sample mean and sample variance, will be considered. Nevertheless, the results are rather disturbing: standard estimators have sizable biases, their variances are (much) larger than usual, and standard confidence intervals do not have the prescribed confidence level any more. Crucial is of course the criterion used to decide whether or not to extend the original sample. Four criteria are applied. In the first three cases, an independent event, the observed sample mean and the observed sample variance, respectively, determine whether or not to double the original sample size. The fourth criterion compares the variances observed in two independent samples; the sample with the highest variance is extended. Only in this fourth case the size of the extension is a random variable. Note that a given criterion is used only once: after the original observation s the final sample size is determined; hence the title of our paper.

In planning a study, the choice of sample size may depend on a variance value based on speculation or obtained from an earlier study. Scientists may wish to use an internal pilot design to protect themselves against an incorrect choice of variance. Such a design involves collecting a portion of the originally planned sample and using it to produce a new variance estimate. This leads to a new power analysis and increasing or decreasing sample size. For any general linear univariate model, with fixed predictors and Gaussian errors, we prove that the uncorrected fixed sample F-statistic is the likelihood ratio test statistic. However, the statistic does not follow an F distribution. Ignoring the discrepancy may inflate test size. We derive and evaluate properties of the components of the likelihood ratio test statistic in order to characterize and quantify the bias. Most notably, the fixed sample size variance estimate becomes biased downward. The bias may inflate test size for any hypothesis test, even if the parameter being tested was not involved in the sample size re-estimation. Furthermore, using fixed sample size methods may create biased confidence intervals for secondary parameters and the variance estimate.

Researchers' interpretations of statistical significance relative to sample size were obtained and analyzed. Fifty-two graduate students responded to a 6-question instrument designed to measure interpretations of the relationships between sample size and treatment effect, sample size and Type I error, and sample size and Type II error. Respondents placed more confidence in results of studies with large sample sizes than in results of studies with small sample sizes, regardless of the criterion on which that confidence was based. A discussion of the mathematical procedure for significance testing shows that respondents were not justified in these interpretations. An improved understanding of the relationship between treatment effect, sample size, and errors of statistical inference is suggested.

We investigated how people use base rates and sample size information when combining data to make overall probability judgments. Participants considered two samples from an animal population in order to estimate the probability of that animal being aggressive. Participants' judgments were influenced by subpopulation base rates when they were provided and linked to specific samples. When samples were not identified as coming from different subpopulations, judgments typically reflected sample size information. We conclude that 1) People can use base rates when combining samples to make an inference; 2) People can correctly use sampling information to determine when to use base rates, and 3) People are able to consider base rate and sample size information at the same time. Additionally, we found that individuals' numeracy correlates with the extent to which base rate and sample size information is used.

The choice of sample size is a fundamental decision in planning an environmental survey. Dale et al. (Ecol. Modelling, 57: 1–10) developed a model for sampling field plots and concluded that: (1) the sample size should be such that the sample skewness and kurtosis fall within certain ranges, (2) a larger sample size is required from a non-normal distribution than from a normal distribution, and (3) sample means from log-normal distributions are biased. We use statistical sampling theory to show that conclusions (2) and (3) are incorrect and that (1) is unnecessary and inefficient. The sample mean from a simple random sample of any population is an unbiased estimate of the population mean; its precision depends on the sample size and variance, but not the skewness or kurtosis. Non-normal populations do not require larger sample sizes, unless they have a greater variance. Dale et al. ignored sampling variation when they used the results of a small number of samples to make general conclusions about sampling methods. They also confounded the effects of distribution and variance when they generated random numbers with inappropriate variances.

While probability sampling has the advantage of permitting unbiased population estimates, many past and existing monitoring schemes do not employ probability sampling. We describe and demonstrate a general procedure for augmenting an existing probability sample with data from nonprobability-based surveys ('found' data). The procedure, first proposed by Overton (1990), uses sampling frame attributes to group the probability and found samples into similar subsets. Subsequently, this similarity is assumed to reflect the representativeness of the found sample for the matching subpopulation. Two methods of establishing similarity and producing estimates are described: pseudo-random and calibration. The pseudo-random method is used when the found sample can contribute additional information on variables already measured for the probability sample, thus increasing the effective sample size. The calibration method is used when the found sample contributes information that is unique to the found observations. For either approach, the found sample data yield observations that are treated as a probability sample, and population estimates are made according to a probability estimation protocol. To demonstrate these approaches, we applied them to found and probability samples of stream discharge data for the southeastern US.

Researchers who plan to draw inferences about the population from a random sample, must first determine how many members of the population should be sampled. To determine this, the researcher must specify the desired precision of the sample estimate, and confidence level associated with this estimate. If the purpose of the random sample is to test a hypothesis, the researcher must specify the desired statistical power for the test, as well as the acceptable risk of rejecting the null hypothesis when it is true (type I error). This essay explains how to determine the sample size for these two types of study. Keywords: sampling error; confidence; level; statistical power; sample size for different sampling techniques; sample size for hypothesis testing

In the first three experiments, we attempted to learn more about subjects' understanding of the importance of sample size by systematically changing aspects of the problems we gave to subjects. In a fourth study, understanding of the effects of sample size was tested as subjects went through a computerassisted training procedure that dealt with random sampling and the sampling distribution of the mean. Subjects used sample size information more appropriately for problems that were stated in terms of the accuracy of the sample average or the center of the sampling distribution than for problems stated in terms of the tails of the sampling distribution. Apparently, people understand that the means of larger samples are more likely to resemble the population mean but not the implications of this fact for the variability of the mean. The fourth experiment showed that although instruction about the sampling distribution of the mean led to better understanding of the effects of sample size, subjects were still unable to make correct inferences about the variability of the mean. The appreciation that people have for some aspects of the law of large numbers does not seem to result from an in-depth understanding of the relation between sample size and variability.

Summary The problem of testing for treatment effect based on binary response data is considered, assuming that the sample size for each experimental unit and treatment combination is random. It is assumed that the sample size follows a distribution that belongs to a parametric family. The uniformly most powerful unbiased tests, which are equivalent to the likelihood ratio tests, are obtained when the probability of the sample size being zero is positive. For the situation where the sample sizes are always positive, the likelihood ratio tests are derived. These test procedures, which are unconditional on the random sample sizes, are useful even when the random sample sizes are not observed. Some examples are presented as illustration.

This paper examines the applicability of recent findings from behavioral economics to military decision making. Army manuals concerning the Military Decision Making Process mention general biases in decision making but do not mention specific biases or specific mechanisms for mitigating bias. Recent research has shed light on specific biases to include: overconfidence, insensitivity to sample size, availability, illusionary correlation, retrievability of instances, escalation, break even, snake bite, fear of regret, and the confirmation bias. The Military Decision Making Process has a long and distinguished record of success. However, there are also examples of military failures due to cognitive bias. These failures include Lee at Gettysburg and McClellan in Virginia. Private industry and some elements of the Army have started to account for these deficiencies through various practices including coaching and training. This paper concludes that the Military Decision Making Process as described in FM 5-0 is deficient in not fully recognizing and accounting for cognitive biases. The process can be improved through several steps. These steps include not only research, education, and training, but also procedural and organizational changes.

When Hotelling's T2 is used as a control statistic for a multivariate normal population with unknown parameters, the statistic is commonly expressed as a function of the sample mean vector and the sample covariance matrix. In this setting, it is well known that as the sample size increases, these sample estimators approach their true population values, as they are consistent estimators. However, it is not well established as to what sample size is needed to assure that this result is approximately true. This is due, in part, to the large number of parameters that must be estimated in the multivariate case. To solve this problem, we present three helpful procedures for determining an effective sample size when using the T2 statistic to monitor the performance of a process.

Advantages of using equal sample sizes in 2 sample tests for means, correlations, and proportions are well known. However, in applied research there are frequently circumstances that limit the size of 1 of the 2 samples. This article draws attention to a simple method of determining, from Cohen's (1988) tables, the effects that this constraint (on the size of one sample) has on the maximum attainable power of these tests. These effects can be extremely serious in the case of small, medium, and even large effect sizes and clearly indicate that availability of a very large 2nd sample may not compensate for a constraint on the size of the 1st sample

Background The p value obtained from a significance test provides no information about the magnitude or importance of the underlying phenomenon. Therefore, additional reporting of effect size is often recommended. Effect sizes are theoretically independent from sample size. Yet this may not hold true empirically: non-independence could indicate publication bias. Methods We investigate whether effect size is independent from sample size in psychological research. We randomly sampled 1,000 psychological articles from all areas of psychological research. We extracted p values, effect sizes, and sample sizes of all empirical papers, and calculated the correlation between effect size and sample size, and investigated the distribution of p values. Results We found a negative correlation of r = −.45 [95% CI: −.53; −.35] between effect size and sample size. In addition, we found an inordinately high number of p values just passing the boundary of significance. Additional data showed that neither implicit nor explicit power analysis could account for this pattern of findings. Conclusion The negative correlation between effect size and samples size, and the biased distribution of p values indicate pervasive publication bias in the entire field of psychology.

When an effect size estimate from an earlier study is available, it is common practice to calculate a sample size for a future study by postulating that the true effect size equals the point estimate from the previous study. Such an approach is methodologically flawed because it ignores the presence of uncertainty in the point estimate. In view of the fact that the sample size needed for a specified power does not vary linearly with effect size, the traditional approach typically leads to less power than is desired for the specified effect. A rigorous method of utilizing information from a previous study when determining the sample size for a chi-squared test is outlined. The approach is able to guarantee that the average power of all experiments in a discipline attains the desired level. Researchers seeking to determine the sample size necessary to replicate an earlier finding, obtained in a pilot study or gleaned from a report in the literature, often decide to base their calculations on an estimate of effect size derived from the initial experiment. The practice of substituting an empirical estimate of effect size in the traditional sample size formula gives rise to a number of problems. Korn (1990) has drawn attention to the poor performance of maximum likelihood estimators in this situation. He notes that the use of any point estimator, without taking into account the uncertainty surrounding the estimate, is likely to generate similar difficulties. It can be shown that the strategy of substituting an empirical estimate of effect size in the traditional sample size formula leads to an average level of power across all experiments that

The sample size necessary for a statistically significant life test has been determined, and a comparison of actual and predicted lives has been made for a desired probability of survival with a given confidence level, statistical error, and coefficient of variation at different stress levels. This approach has been used for cases where data follow a log-normal distribution. On the basis of this approximation, it is possible to estimate the lower tolerance limit using a minimum number of samples. It has also been concluded that, whatever the sample size, the estimate always results in some error, R0, which mainly depends on confidence level, probability of survival, and stress level, and that, beyond a specified sample size, fatigue life estimation is independent of sample size.

In sample surveys, quite often, the sample size becomes a random variable. This happens mainly in two situations, namely, (i) when the planned sample size may be fixed but the realized sample size may be random, and (ii) when planned sample size itself may be random. In the present investigation we propose a method of estimation which is quite efficient in both these situations.

Most people do not seem to realize that an extreme value for a sample statistic is more likely the smaller the sample. However, Harvard undergraduates, who have satisfied a Quantitative Reasoning Requirement, do much better on some versions of sample size questions than previous populations have. Nevertheless, these results show that subjects' difficulty in selecting the correct answer to questions about the relationship between sample statistic and sample size is not related to the presence or wording of an answer choice that states there is no relationship. Further, formally identical versions of problems yield quite different rates of correct answers.

The present research focused on sample size as one factor which may mediate the efficacy of sample-based consensus information. In one experiment, subjects confronted with two conflicting base rates relied on the one derived from the larger sample when making individual and population predictions. In a second experiment, subjects for whom consensus percentages were discrepant with expectations assumed that these base rates were derived from a smaller sample than did subjects for whom the percentages were congruent with expectations. These results were discussed in the context of the literature on consensus, prediction, and attribution.

The paper considers inference about an infinite population supposedly produced by a system of chance causes. The existing universe is regarded as a sample from this parent infinite population. Whatever the sampling design, the sample arithmetic mean that is based on the effective sample size, is found to be the minimum variance unbiased estimator of the mean of the infinite population, and its existence is linked with each of two separate sets of conditions required for its unbiasedness. When the parent infinite population is assumed to be normally distributed and when the effective sample size varies from one possible sample to another, the skewness and kurtosis coefficients of the sampling distribution of the mean firmly indicate that normality is not automatically achieved and is achieved only for those sampling procedures for which the effective sample size remains constant. Brief comments are made on the extent of appropriateness of confidence interval statements for the mean of the parent infinite p...