Search

"What should be the sample size for my study?" is a common question in the minds of every research at some point of the research cycle. Answering this question with confident is tough even for a seasoned researcher. Sample size determination, an important aspect of sampling design consideration of a study, is a factor which directly influences the internal and external validity of the study. Unless the sample size is of adequate size, the results of the study cannot be justified. Conducting a study in too small sample size or too large sample size have ethical, scientific, practical, and economic strings attached to it and have detrimental effects in the research outcomes. A myriad of factors including the study design, type of power analysis, sampling technique employed, and acceptable limits of error fixed play a decisive role in estimating the sample size. However, the advent of free to use software and websites for sample size estimation has actually diluted or sometimes complicated the whole process of sample size estimation as important factors or assumptions related to sample size are overlooked. Engaging a professional biostatistician from the very beginning of the research process would be a wise decision while conducting research. This article highlights the important concepts related to sample size estimation with emphasis on factors which influences it. [ABSTRACT FROM AUTHOR]

“What should be the sample size for my study?” is a common question in the minds of every research at some point of the research cycle. Answering this question with confident is tough even for a seasoned researcher. Sample size determination, an important aspect of sampling design consideration of a study, is a factor which directly influences the internal and external validity of the study. Unless the sample size is of adequate size, the results of the study cannot be justified. Conducting a study in too small sample size or too large sample size have ethical, scientific, practical, and economic strings attached to it and have detrimental effects in the research outcomes. A myriad of factors including the study design, type of power analysis, sampling technique employed, and acceptable limits of error fixed play a decisive role in estimating the sample size. However, the advent of free to use software and websites for sample size estimation has actually diluted or sometimes complicated the whole process of sample size estimation as important factors or assumptions related to sample size are overlooked. Engaging a professional biostatistician from the very beginning of the research process would be a wise decision while conducting research. This article highlights the important concepts related to sample size estimation with emphasis on factors which influences it.

Studies of sex chromosome systems at early stages of divergence are key to understanding the initial process and underlying causes of recombination suppression. However, identifying signatures of divergence in homomorphic sex chromosomes can be challenging due to high levels of sequence similarity between the X and the Y. Variations in methodological precision and underlying data can make all the difference between detecting subtle divergence patterns or missing them entirely. Recent efforts to test for X‐Y sequence differentiation in the guppy have led to contradictory results. Here, we apply different analytical methodologies to the same data set to test for the accuracy of different approaches in identifying patterns of sex chromosome divergence in the guppy. Our comparative analysis reveals that the most substantial source of variation in the results of the different analyses lies in the reference genome used. Analyses using custom‐made genome assemblies for the focal population or species successfully recover a signal of divergence across different methodological approaches. By contrast, using the distantly related Xiphophorus reference genome results in variable patterns, due to both sequence evolution and structural variations on the sex chromosomes between the guppy and Xiphophorus. Changes in mapping and filtering parameters can additionally introduce noise and obscure the signal. Our results illustrate how analytical differences can alter perceived results and we highlight best practices for the study of nascent sex chromosomes. [ABSTRACT FROM AUTHOR]

The evolution of autism diagnosis, from its discovery to its current delineation using standardized instruments, has been paralleled by a steady increase in its prevalence and heterogeneity. In clinical settings, the diagnosis of autism is now too vague to specify the type of support required by the concerned individuals. In research, the inclusion of individuals categorically defined by over‐inclusive, polythetic criteria in autism cohorts results in a population whose heterogeneity runs contrary to the advancement of scientific progress. Investigating individuals sharing only a trivial resemblance produces a large‐scale type‐2 error (not finding differences between autistic and dominant population) rather than detecting mechanistic differences to explain their phenotypic divergences. The dimensional approach of autism proposed to cure the disease of its categorical diagnosis is plagued by the arbitrariness of the dimensions under study. Here, we argue that an emphasis on the reliability rather than specificity of diagnostic criteria and the misuse of diagnostic instruments, which ignore the recognition of a prototype, leads to confound autism with the entire range of neurodevelopmental conditions and personality variants. We propose centering research on cohorts in which individuals are selected based on their expert judged prototypicality to advance the theoretical and practical pervasive issues pertaining to autism diagnostic thresholds. Reversing the current research strategy by giving more weight to specificity than reliability should increase our ability to discover the mechanisms of autism. Lay Summary: Scientific research into the causes of autism and its mechanisms is carried out on large cohorts of people who are less and less different from the general population. This historical trend may explain the poor harvest of results obtained. Services and intervention are provided according to a diagnosis that now encompasses extremely different individuals. Last, we accept as a biological reality the constant increase over the years in the proportion of autistic people among the general population. These drifts are made possible by the attribution of a diagnosis of autism to people who meet vague criteria, rather than to people who experienced clinicians recognize as autistic. We propose to change our research strategy by focusing on the study of the latter, fewer in number, but more representative of the "prototype" of autism. To do this, it is necessary to clearly distinguish the population on which the research is carried out from that to which we provide support. People must receive services according to their needs, and not according to the clarity of their diagnosis. [ABSTRACT FROM AUTHOR]

When comparing two independent groups, psychology researchers commonly use Student’s 't'-tests. Assumptions of normality and homogeneity of variance underlie this test. More often than not, when these conditions are not met, Student’s 't'-test can be severely biased and lead to invalid statistical inferences. Moreover, we argue that the assumption of equal variances will seldom hold in psychological research, and choosing between Student’s 't'-test and Welch’s 't'-test based on the outcomes of a test of the equality of variances often fails to provide an appropriate answer. We show that the Welch’s 't'-test provides a better control of Type 1 error rates when the assumption of homogeneity of variance is not met, and it loses little robustness compared to Student’s 't'-test when the assumptions are met. We argue that Welch’s 't'-test should be used as a default strategy.

Many articles in the surgical literature were faulted for committing type 2 error, or concluding no difference when the study was "underpowered". However, it is unknown if the current power standard of 0.8 is reasonable in surgical science. PubMed was searched for abstracts published in Surgery, JAMA Surgery, and Annals of Surgery and from January 1, 2012 to December 31, 2016, with Medical Subject Heading terms of randomized controlled trial (RCT) or observational study (OBS) and limited to humans were included (n = 403). Articles were excluded if all reported findings were statistically significant (n = 193), or if presented data were insufficient to calculate power (n = 141). A total of 69 manuscripts (59 RCTs and 10 OBSs) were assessed. Overall, the median power was 0.16 (interquartile range [IQR] 0.08-0.32). The median power was 0.16 for RCTs (IQR 0.08-0.32) and 0.14 for OBSs (IQR 0.09-0.22). Only 4 studies (5.8%) reached or exceeded the current 0.8 standard. Two-thirds of our study sample had an a priori power calculation (n = 41). High-impact surgical science was routinely unable to reach the arbitrary power standard of 0.8. The academic surgical community should reconsider the power threshold as it applies to surgical investigations. We contend that the blueprint for the redesign should include benchmarking the power of articles on a gradient scale, instead of aiming for an unreasonable threshold. [ABSTRACT FROM AUTHOR]

Determining the appropriate sample size for a study, whatever be its type, is a fundamental aspect of biomedical research. An adequate sample ensures that the study will yield reliable information, regardless of whether the data ultimately suggests a clinically important difference between the interventions or elements being studied. The probability of Type 1 and Type 2 errors, the expected variance in the sample and the effect size are the essential determinants of sample size in interventional studies. Any method for deriving a conclusion from experimental data carries with it some risk of drawing a false conclusion. Two types of false conclusion may occur, called Type 1 and Type 2 errors, whose probabilities are denoted by the symbols σ and β. A Type 1 error occurs when one concludes that a difference exists between the groups being compared when, in reality, it does not. This is akin to a false positive result. A Type 2 error occurs when one concludes that difference does not exist when, in reality, a difference does exist, and it is equal to or larger than the effect size defined by the alternative to the null hypothesis. This may be viewed as a false negative result. When considering the risk of Type 2 error, it is more intuitive to think in terms of power of the study or (1 − β). Power denotes the probability of detecting a difference when a difference does exist between the groups being compared. Smaller α or larger power will increase sample size. Conventional acceptable values for power and α are 80% or above and 5% or below, respectively, when calculating sample size. Increasing variance in the sample tends to increase the sample size required to achieve a given power level. The effect size is the smallest clinically important difference that is sought to be detected and, rather than statistical convention, is a matter of past experience and clinical judgment. Larger samples are required if smaller differences are to be detected. Although the principles are long known, historically, sample size determination has been difficult, because of relatively complex mathematical considerations and numerous different formulas. However, of late, there has been remarkable improvement in the availability, capability, and user-friendliness of power and sample size determination software. Many can execute routines for determination of sample size and power for a wide variety of research designs and statistical tests. With the drudgery of mathematical calculation gone, researchers must now concentrate on determining appropriate sample size and achieving these targets, so that study conclusions can be accepted as meaningful.

Sample-size estimation based on the traditional method of statistical significance is not appropriate for a study designed to make an inference about real-world significance, which requires interpretation of magnitude of an outcome. I present here a spreadsheet using two new methods for estimating sample size for such studies, based on acceptable uncertainty defined either by the width of the confidence interval or by error rates for a clinical or practical decision arising from the study. The new methods require sample sizes approximately one-third those of the traditional method, which is included in the spreadsheet. The following issues are also addressed in this article: choice of smallest effect, sample size with various designs, sample size "on the fly", dealing with suboptimal sample size, effect of validity and reliability of dependent and predictor variables, sample size for comparison of subgroups, sample size for individual differences and responses, sample size when adjusting for subgroups of unequal size, sample size for more than one important effect, the number of repeated observations in single-subject studies, sample sizes for measurement studies and case series, and estimation of sample size by simulation. [ABSTRACT FROM AUTHOR]

Determining the appropriate sample size for a study, whatever be its type, is a fundamental aspect of biomedical research. An adequate sample ensures that the study will yield reliable information, regardless of whether the data ultimately suggests a clinically important difference between the interventions or elements being studied. The probability of Type 1 and Type 2 errors, the expected variance in the sample and the effect size are the essential determinants of sample size in interventional studies. Any method for deriving a conclusion from experimental data carries with it some risk of drawing a false conclusion. Two types of false conclusion may occur, called Type 1 and Type 2 errors, whose probabilities are denoted by the symbols σ and β. A Type 1 error occurs when one concludes that a difference exists between the groups being compared when, in reality, it does not. This is akin to a false positive result. A Type 2 error occurs when one concludes that difference does not exist when, in reality, a difference does exist, and it is equal to or larger than the effect size defined by the alternative to the null hypothesis. This may be viewed as a false negative result. When considering the risk of Type 2 error, it is more intuitive to think in terms of power of the study or (1 - β). Power denotes the probability of detecting a difference when a difference does exist between the groups being compared. Smaller α or larger power will increase sample size. Conventional acceptable values for power and α are 80% or above and 5% or below, respectively, when calculating sample size. Increasing variance in the sample tends to increase the sample size required to achieve a given power level. The effect size is the smallest clinically important difference that is sought to be detected and, rather than statistical convention, is a matter of past experience and clinical judgment. Larger samples are required if smaller differences are to be detected. Although the principles are long known, historically, sample size determination has been difficult, because of relatively complex mathematical considerations and numerous different formulas. However, of late, there has been remarkable improvement in the availability, capability, and user-friendliness of power and sample size determination software. Many can execute routines for determination of sample size and power for a wide variety of research designs and statistical tests. With the drudgery of mathematical calculation gone, researchers must now concentrate on determining appropriate sample size and achieving these targets, so that study conclusions can be accepted as meaningful. [ABSTRACT FROM AUTHOR]

Two recent studies published in The Lancet (Autier et al. (2013) Lancet Diabetes Endocrinol2, 76–89 and Bolland et al. (2014) Lancet Diabetes Endocrinol2, 307–320) have concluded that low levels of vitamin D are not a cause but a consequence of ill health brought about by reduced exposure to the sun, an association known as ‘reverse causality’. The scientific evidence and reasoning for these conclusions are examined here and found to be faulty. A null result in a clinical trial of vitamin D in adults need not lead to a conclusion of reverse causation when low vitamin D is found in observational studies of the same disease earlier in life. To assume an explanation of reverse causality has close similarities with type 2 statistical error.For example, a null result in providing vitamin D for treatment of adult bones that are deformed in the pattern of the rachitic rosary would not alter the observation that lack of vitamin D can cause rickets in childhood and may have lasting consequences if not cured with vitamin D. Other examples of diseases considered on a lifetime basis from conception to adulthood are used to further illustrate the issue, which is evidently not obvious and is far from trivial.It is concluded that deficiency of vitamin D in cohort studies, especially at critical times such as pregnancy and early life, can be the cause of a number of important diseases. Denial of the possible benefits of vitamin D, as suggested by insistent interpretation of studies with reverse causation, may lead to serious harms, some of which are listed. [ABSTRACT FROM PUBLISHER]

In omic research, such as genome wide association studies, researchers seek to repeat their results in other datasets to reduce false positive findings and thus provide evidence for the existence of true associations. Unfortunately this standard validation approach cannot completely eliminate false positive conclusions, and it can also mask many true associations that might otherwise advance our understanding of pathology. These issues beg the question: How can we increase the amount of knowledge gained from high throughput genetic data? To address this challenge, we present an approach that complements standard statistical validation methods by drawing attention to both potential false negative and false positive conclusions, as well as providing broad information for directing future research. The Diverse Convergent Evidence approach (DiCE) we propose integrates information from multiple sources (omics, informatics, and laboratory experiments) to estimate the strength of the available corroborating evidence supporting a given association. This process is designed to yield an evidence metric that has utility when etiologic heterogeneity, variable risk factor frequencies, and a variety of observational data imperfections might lead to false conclusions. We provide proof of principle examples in which DiCE identified strong evidence for associations that have established biological importance, when standard validation methods alone did not provide support. If used as an adjunct to standard validation methods this approach can leverage multiple distinct data types to improve genetic risk factor discovery/validation, promote effective science communication, and guide future research directions. [ABSTRACT FROM AUTHOR]

Researchers often use very abbreviated (e.g., 1-item, 2-item) measures of personality traits due to their convenience and ease of use as well as the belief that such measures can adequately capture an individual's personality. Using data from 2 samples (N = 437 employees, N = 355 college students), we show that this practice, particularly the use of single-item measures, can lead researchers to substantially underestimate the role that personality traits play in influencing important behaviors and thereby overestimate the role played by new constructs. That is, the use of very short measures of personality may substantially increase both the Type 1 and Type 2 error rates. We argue that even slightly longer measures can substantially increase the validity of research findings without significant inconvenience to the researcher or research participants. [ABSTRACT FROM AUTHOR]

Null models of species co-occurrence are widely used to infer the existence of various ecological processes. Here we investigate the susceptibility of the most commonly used of these models (the C-score in conjunction with the sequential swap algorithm) to type 1 and type 2 errors. To do this we use simulated datasets with a range of numbers of sites, species and coefficients of variation (CV) in species abundance. We find that this model is particularly susceptible to type 1 errors when applied to large matrices and those with low CV in species abundance. As expected, type 2 error rates decrease with increasing numbers of sites and species, although they increase with increasing CV in species abundance. Despite this, power remains acceptable over a wide range of parameter combinations. The susceptibility of this analytical method to type 1 errors indicates that many previous studies may have incorrectly reported the existence of deterministic patterns of species co-occurrence. We demonstrate that in order to overcome the problem of high type 1 error rates, the number of swaps used to generate null distributions for smaller matrices needs to be increased to over 50,000 swaps (well beyond the 5000 commonly used in published analyses and the 30,000 suggested by ). We also show that this approach reduces type 1 error rates in real datasets. However, even using this solution, larger datasets still suffer from high type 1 error rates. Such datasets therefore require the use of very large numbers of swaps, which calls for improvements in the most commonly used software. In general, users of this powerful analytical method must be aware that they need surprisingly large numbers of swaps to obtain unbiased estimates of structuring in biotic communities. [Copyright &y& Elsevier]

Joint destruction is a measure for RA severity that is accurate, sensitive and reflective of the cumulative disease burden. Risk factors for this outcome measure may be used to arrive at individualized treatment strategies. Currently, relatively few risk factors for joint destruction are known. New risk factors, genetic risk factors in particular, may have relatively small effects on the rate of joint destruction. A sensitive determination of joint damage is then crucial in order to identify these risk factors and will reduce the risk on type 2 errors. The present article addresses the question how the rate of joint destruction is ideally measured. Different methods are discussed and suggestions for corrections of factors that affect the natural course of joint destruction, such as applied treatment strategies, are made. It is concluded that a precise estimation of the rate of radiological joint destruction is obtained by using quantitative and validated scoring methods as well as repetitive measurements over time in order to reduce within patient variation. [ABSTRACT FROM PUBLISHER]

The social constructs and methodological principles embodied in the Maryland Scientific Methods Scale (SMS), comprising part of the Campbell Collaboration in Crime and Justice assessment protocol, induce a series of biases in the evaluation of evidence of crime prevention policy interventions that focus on collective social phenomena, such as communities. Applying these principles leads to negative conclusions about effectiveness; yet their inherent ‘anti-social’ bias may induce Type II error with regard to the desirability of ‘social’ interventions to reduce crime. Policy-making is poorly served as a result. This point is illustrated, first, through a scrutiny of the social constructs used, including those that typify treatments, institutional settings and units of analysis. These are seen as being constructed in a way that is congenial to the underlying methodological issue of ‘control’ but that constitute nevertheless a distorted definition of the governance issues involved in crime reduction in community settings. A model more appropriate for evaluating voluntaristic action in civil society is needed. Second, it is suggested that this methodological bias arises particularly in policy interventions and change programmes that address issues concerning the ‘collective efficacy’ of local communities in reducing crime. An empirical exemplification of these arguments is presented with reference to a completed evaluation research study (Foster and Hope, 1993). [ABSTRACT FROM AUTHOR]

Power tells us the probability of rejecting the null hypothesis for an effect of a given size and helps us select an appropriate design prior to running the experiment. The key to computing power for an effect is determining the size of the effect. We describe a general approach for sizing effects that covers a wide variety of designs including factorials with categorical levels, response surfaces, mixtures and crossed designs. Copyright © 2001 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The evolution of autism diagnosis, from its discovery to its current delineation using standardized instruments, has been paralleled by a steady increase in its prevalence and heterogeneity. In clinical settings, the diagnosis of autism is now too vague to specify the type of support required by the concerned individuals. In research, the inclusion of individuals categorically defined by over-inclusive, polythetic criteria in autism cohorts results in a population whose heterogeneity runs contrary to the advancement of scientific progress. Investigating individuals sharing only a trivial resemblance produces a large-scale type-2 error (not finding differences between autistic and dominant population) rather than detecting mechanistic differences to explain their phenotypic divergences. The dimensional approach of autism proposed to cure the disease of its categorical diagnosis is plagued by the arbitrariness of the dimensions under study. Here, we argue that an emphasis on the reliability rather than specificity of diagnostic criteria and the misuse of diagnostic instruments, which ignore the recognition of a prototype, leads to confound autism with the entire range of neurodevelopmental conditions and personality variants. We propose centering research on cohorts in which individuals are selected based on their expert judged prototypicality to advance the theoretical and practical pervasive issues pertaining to autism diagnostic thresholds. Reversing the current research strategy by giving more weight to specificity than reliability should increase our ability to discover the mechanisms of autism. LAY SUMMARY: Scientific research into the causes of autism and its mechanisms is carried out on large cohorts of people who are less and less different from the general population. This historical trend may explain the poor harvest of results obtained. Services and intervention are provided according to a diagnosis that now encompasses extremely different individuals. Last, we accept as a biological reality the constant increase over the years in the proportion of autistic people among the general population. These drifts are made possible by the attribution of a diagnosis of autism to people who meet vague criteria, rather than to people who experienced clinicians recognize as autistic. We propose to change our research strategy by focusing on the study of the latter, fewer in number, but more representative of the "prototype" of autism. To do this, it is necessary to clearly distinguish the population on which the research is carried out from that to which we provide support. People must receive services according to their needs, and not according to the clarity of their diagnosis., (© 2021 International Society for Autism Research and Wiley Periodicals LLC.)

The paper describes historic origins of p values via the work of Fisher, and the competing approach by Neyman and Pearson. Concepts of type 1 and type 2 errors, false positive rates, power, and prevalence are also defined, and the merger of the two approaches via the Null Hypothesis Significance Test. The relationship between p values and false detection rate is discussed. The reproducibility of p values is described. The current controversy over the use of p values and significance tests is introduced. • Historical review of the concept of p values. • Relationship between common concepts such as False Positive Rates and p values and type 1 and type 2 errors. • Reproducibility of p values. • Current controversy over use of p values. [ABSTRACT FROM AUTHOR]