Search

Theoretical and empirical evidence indicates that low external validity due to rigorous standardization of study populations is a cause of poor replicability in animal research. Here, we report a multi-laboratory study to test whether heterogenization of study populations by using animals from different breeders increases the replicability of results from single-laboratory studies. We found that heterogenization by breeder was not potent enough to increase external validity sufficiently to improve replicability.

Studies in mice have shown that less aversive handling methods (e.g. tunnel or cup handling) can reduce behavioural measures of anxiety in comparison to picking mice up by their tail. Despite such evidence, tail handling continues to be used routinely. Besides resistance to change accustomed procedures, this may also be due to the fact that current evidence in support of less aversive handling is mostly restricted to effects of extensive daily handling, which may not apply to routine husbandry practices. The aim of our study was to assess whether, and to what extent, different handling methods during routine husbandry induce differences in behavioural and physiological measures of stress in laboratory mice. To put the effects of handling method in perspective with chronic stress, we compared handling methods to a validated paradigm of unpredictable chronic mild stress (UCMS). We housed mice of two strains (Balb/c and C57BL/6) and both sexes either under standard laboratory conditions (CTRL) or under UCMS. Half of the animals from each housing condition were tail handled and half were tunnel handled twice per week, once during a cage change and once for a routine health check. We found strain dependent effects of handling method on behavioural measures of anxiety: tunnel handled Balb/c mice interacted with the handler more than tail handled conspecifics, and tunnel handled CTRL mice showed increased open arm exploration in the elevated plus-maze. Mice undergoing UCMS showed increased plasma corticosterone levels and reduced sucrose preference. However, we found no effect of handling method on these stress-associated measures. Our results therefore indicate that routine tail handling can affect behavioural measures of anxiety, but may not be a significant source of chronic husbandry stress. Our results also highlight strain dependent responses to handling methods.

The phenotype of an organism results from its genotype and the influence of the environment throughout development. Even when using animals of the same genotype, independent studies may test animals of different phenotypes, resulting in poor replicability due to genotype-by-environment interactions. Thus, genetically defined strains of mice may respond differently to experimental treatments depending on their rearing environment. However, the extent of such phenotypic plasticity and its implications for the replicability of research findings have remained unknown. Here, we examined the extent to which common environmental differences between animal facilities modulate the phenotype of genetically homogeneous (inbred) mice. We conducted a comprehensive multicentre study, whereby inbred C57BL/6J mice from a single breeding cohort were allocated to and reared in 5 different animal facilities throughout early life and adolescence, before being transported to a single test laboratory. We found persistent effects of the rearing facility on the composition and heterogeneity of the gut microbial community. These effects were paralleled by persistent differences in body weight and in the behavioural phenotype of the mice. Furthermore, we show that environmental variation among animal facilities is strong enough to influence epigenetic patterns in neurons at the level of chromatin organisation. We detected changes in chromatin organisation in the regulatory regions of genes involved in nucleosome assembly, neuronal differentiation, synaptic plasticity, and regulation of behaviour. Our findings demonstrate that common environmental differences between animal facilities may produce facility-specific phenotypes, from the molecular to the behavioural level. Furthermore, they highlight an important limitation of inferences from single-laboratory studies and thus argue that study designs should take environmental background into account to increase the robustness and replicability of findings.

The phenotype of an organism results from its genotype and the influence of the environment throughout development. Even when using animals of the same genotype, independent studies may test animals of different phenotypes, resulting in poor replicability due to genotype-by-environment interactions 1–4. Thus, genetically defined strains of mice may respond differently to experimental treatments depending on their rearing environment 5. However, the extent of such phenotypic plasticity and its implications for the replicability of research findings have remained unknown. Here, we examined the extent to which common environmental differences between rearing facilities modulate the phenotype of genetically homogeneous (inbred) mice. We conducted a comprehensive multi-center study, where inbred mice from the same breeding stock were reared in five different facilities throughout early life and adolescence, before being transported to a single test laboratory. We found persistent effects of rearing facility on the composition and heterogeneity of the gut microbial community. These effects were paralleled by persistent differences in body weight and in the behavioural phenotype of the mice. Furthermore, we show that common variation among rearing facilities is strong enough to influence epigenetic patterns in neurons at the level of chromatin organization. We detected changes in chromatin organization in the regulatory regions of genes involved in nucleosome assembly, neuronal differentiation, synaptic plasticity and regulation of behavior. Our findings demonstrate that common environmental differences between rearing facilities may produce facility-specific phenotypes, from the molecular to the behavioural level. We expect our findings to stimulate further research into the mechanisms and drivers of these epigenetic changes mediated by the laboratory environment. Furthermore, they highlight an important limitation of inferences from single-laboratory studies and a need to account for the animals’ environmental background in study design to produce robust and replicable findings.

The phenotype of an organism results from its genotype and the influence of the environment throughout development. Even when using animals of the same genotype, independent studies may test animals of different phenotypes, resulting in poor replicability due to genotype-by-environment interactions. Thus, genetically defined strains of mice may respond differently to experimental treatments depending on their rearing environment. However, the extent of such phenotypic plasticity and its implications for the replicability of research findings have remained unknown. Here, we examined the extent to which common environmental differences between animal facilities modulate the phenotype of genetically homogeneous (inbred) mice. We conducted a comprehensive multicentre study, whereby inbred C57BL/6J mice from a single breeding cohort were allocated to and reared in 5 different animal facilities throughout early life and adolescence, before being transported to a single test laboratory. We found persistent effects of the rearing facility on the composition and heterogeneity of the gut microbial community. These effects were paralleled by persistent differences in body weight and in the behavioural phenotype of the mice. Furthermore, we show that environmental variation among animal facilities is strong enough to influence epigenetic patterns in neurons at the level of chromatin organisation. We detected changes in chromatin organisation in the regulatory regions of genes involved in nucleosome assembly, neuronal differentiation, synaptic plasticity, and regulation of behaviour. Our findings demonstrate that common environmental differences between animal facilities may produce facility-specific phenotypes, from the molecular to the behavioural level. Furthermore, they highlight an important limitation of inferences from single-laboratory studies and thus argue that study designs should take environmental background into account to increase the robustness and replicability of findings., PLoS Biology, 20 (10), ISSN:1544-9173, ISSN:1545-7885

The influence of protocol standardization between laboratories on their replicability of preclinical results has not been addressed in a systematic way. While standardization is considered good research practice as a means to control for undesired external noise (i.e., highly variable results), some reports suggest that standardized protocols may lead to idiosyncratic results, thus undermining replicability. Through the EQIPD consortium, a multi-lab collaboration between academic and industry partners, we aimed to elucidate parameters that impact the replicability of preclinical animal studies. To this end, 3 experimental protocols were implemented across 7 laboratories. The replicability of results was determined using the distance travelled in an open field after administration of pharmacological compounds known to modulate locomotor activity (MK-801, diazepam, and clozapine) in C57BL/6 mice as a worked example. The goal was to determine whether harmonization of study protocols across laboratories improves the replicability of the results and whether replicability can be further improved by systematic variation (heterogenization) of 2 environmental factors (time of testing and light intensity during testing) within laboratories. Protocols were tested in 3 consecutive stages and differed in the extent of harmonization across laboratories and standardization within laboratories: stage 1, minimally aligned across sites (local protocol); stage 2, fully aligned across sites (harmonized protocol) with and without systematic variation (standardized and heterogenized cohort); and stage 3, fully aligned across sites (standardized protocol) with a different compound. All protocols resulted in consistent treatment effects across laboratories, which were also replicated within laboratories across the different stages. Harmonization of protocols across laboratories reduced between-lab variability substantially compared to each lab using their local protocol. In contrast, the environmental factors chosen to introduce systematic variation within laboratories did not affect the behavioral outcome. Therefore, heterogenization did not reduce between-lab variability further compared to the harmonization of the standardized protocol. Altogether, these findings demonstrate that subtle variations between lab-specific study protocols may introduce variation across independent replicate studies even after protocol harmonization and that systematic heterogenization of environmental factors may not be sufficient to account for such between-lab variation. Differences in replicability of results within and between laboratories highlight the ubiquity of study-specific variation due to between-lab variability, the importance of transparent and fine-grained reporting of methodologies and research protocols, and the importance of independent study replication.

The credibility of scientific research has been seriously questioned by the widely claimed “reproducibility crisis”. In light of this crisis, there is a growing awareness that the rigorous standardisation of experimental conditions may contribute to poor reproducibility of animal studies. Instead, systematic heterogenisation has been proposed as a tool to enhance reproducibility, but a real-life test across multiple independent laboratories is still pending. The aim of this study was therefore to test whether heterogenisation of experimental conditions by using multiple experimenters improves the reproducibility of research findings compared to standardised conditions with only one experimenter. To this end, we replicated the same animal experiment in 3 independent laboratories, each employing both a heterogenised and a standardised design. Whereas in the standardised design, all animals were tested by a single experimenter; in the heterogenised design, 3 different experimenters were involved in testing the animals. In contrast to our expectation, the inclusion of multiple experimenters in the heterogenised design did not improve the reproducibility of the results across the 3 laboratories. Interestingly, however, a variance component analysis indicated that the variation introduced by the different experimenters was not as high as the variation introduced by the laboratories, probably explaining why this heterogenisation strategy did not bring the anticipated success. Even more interestingly, for the majority of outcome measures, the remaining residual variation was identified as an important source of variance accounting for 41% (CI95 [34%, 49%]) to 72% (CI95 [58%, 88%]) of the observed total variance. Despite some uncertainty surrounding the estimated numbers, these findings argue for systematically including biological variation rather than eliminating it in animal studies and call for future research on effective improvement strategies.

Poor reproducibility is considered a serious problem in laboratory animal research, with important scientific, economic, and ethical implications. One possible source of conflicting findings in laboratory animal research are environmental differences between animal facilities combined with rigorous environmental standardization within studies. Due to phenotypic plasticity, study-specific differences in environmental conditions during development can induce differences in the animals’ responsiveness to experimental treatments, thereby contributing to poor reproducibility of experimental results. Here, we studied how variation in weaning age (14–30 days) and housing conditions (single versus group housing) affects the phenotype of SWISS mice as measured by a range of behavioral and physiological outcome variables. Weaning age, housing conditions, and their interaction had little effect on the development of stereotypies, as well as on body weight, glucocorticoid metabolite concentrations, and behavior in the elevated plus-maze and open field test. These results are surprising and partly in conflict with previously published findings, especially with respect to the effects of early weaning. Our results thus question the external validity of previous findings and call for further research to identify the sources of variation between replicate studies and study designs that produce robust and reproducible experimental results.

Behavioural tests to assess affective states are widely used in human research and have recently been extended to animals. These tests assume that affective state influences cognitive processing, and that animals in a negative affective state interpret ambiguous information as expecting a negative outcome (displaying a negative cognitive bias). Most of these tests however, require long discrimination training. The aim of the study was to validate an exploration based cognitive bias test, using two different handling methods, as previous studies have shown that standard tail handling of mice increases physiological and behavioural measures of anxiety compared to cupped handling. Therefore, we hypothesised that tail handled mice would display a negative cognitive bias. We handled 28 female CD-1 mice for 16 weeks using either tail handling or cupped handling. The mice were then trained in an eight arm radial maze, where two adjacent arms predicted a positive outcome (darkness and food), while the two opposite arms predicted a negative outcome (no food, white noise and light). After six days of training, the mice were also given access to the four previously unavailable intermediate ambiguous arms of the radial maze and tested for cognitive bias. We were unable to validate this test, as mice from both handling groups displayed a similar pattern of exploration. Furthermore, we examined whether maze exploration is affected by the expression of stereotypic behaviour in the home cage. Mice with higher levels of stereotypic behaviour spent more time in positive arms and avoided ambiguous arms, displaying a negative cognitive bias. While this test needs further validation, our results indicate that it may allow the assessment of affective state in mice with minimal training— a major confound in current cognitive bias paradigms.

Cognitive processes are influenced by underlying affective states, and tests of cognitive bias have recently been developed to assess the valence of affective states in animals. These tests are based on the fact that individuals in a negative affective state interpret ambiguous stimuli more pessimistically than individuals in a more positive state. Using two strains of mice we explored whether unpredictable chronic mild stress (UCMS) can induce a negative judgement bias and whether variation in the expression of stereotypic behaviour is associated with variation in judgement bias. Sixteen female CD-1 and 16 female C57BL/6 mice were trained on a tactile conditional discrimination test with grade of sandpaper as a cue for differential food rewards. Once they had learned the discrimination, half of the mice were subjected to UCMS for three weeks to induce a negative affective state. Although UCMS induced a reduced preference for the higher value reward in the judgement bias test, it did not affect saccharine preference or hypothalamic–pituitary–adrenal (HPA) activity. However, UCMS affected responses to ambiguous (intermediate) cues in the judgement bias test. While control mice showed a graded response to ambiguous cues, UCMS mice of both strains did not discriminate between ambiguous cues and tended to show shorter latencies to the ambiguous cues and the negative reference cue. UCMS also increased bar-mouthing in CD-1, but not in C57BL/6 mice. Furthermore, mice with higher levels of stereotypic behaviour made more optimistic choices in the judgement bias test. However, no such relationship was found for stereotypic bar-mouthing, highlighting the importance of investigating different types of stereotypic behaviour separately.