Analysing microbiome intervention studies: Does choice of a statistical method affect biological interpretation?

Background The diet plays a major role in shaping gut microbiome composition and function in both humans and animals, and dietary intervention trials are often used to investigate and understand these effects. A plethora of statistical methods for analysing differential abundance of microbial taxa exists, and new methods are constantly being developed, but there is a lack of benchmarking studies and clear consensus on the best multivariate statistical practices. This makes it hard for a biologist to decide which method to use. Results We compared the outcomes of a wide range of ANOVA-like statistical methods to explore to what extent the choice of method affects the biological inferences made. The comparison is based on five published dietary intervention studies representing different subjects and study designs. We found that the methods producing outputs at the community level were in agreement regarding both effect size and statistical significance. However, the methods that provided ranking of operational taxonomic units (OTUs) gave incongruent results, implying that the choice of method is likely to influence the biological interpretations. Conclusions We concluded that inferences at the OTU level should be made with caution, and that a combination of several statistical methods allows to interpret the outcomes with higher confidence and simultaneously account for the limitations of each method.