Search

Biomedical research accuracy and relevance for improving healthcare are increasingly identified as costly problems. Basic research data quality, reporting and methodology, and reproducibility are common factors implicated in this challenge. Preclinical models of disease and therapy, largely conducted in rodents, have known deficiencies in replicating most human conditions. Their translation to human results is acknowledged to be poor for decades. Clinical data quality and quantity is also recognized as deficient; gold standard randomized clinical trials are expensive. Few solid conclusions from clinical studies are replicable and many remain unpublished. The translational pathway from fundamental biomedical research through to innovative solutions handed to clinical practitioners is therefore highly inefficient and costly in terms of wasted resources, early claims from fundamental discoveries never witnessed in humans, and few new, improved solutions available clinically for myriad diseases. Improving this biomedical research strategy and resourcing for reliability, translational relevance, reproducibility and clinical impact requires careful analysis and consistent enforcement at both funding and peer review levels.

A spectre is haunting biomedical research: It appears that a substantial fraction of published research results cannot be reproduced, while spectacularly successful novel treatments developed in experimental models of disease too often fail in clinical trials. A reproducibility crisis has been proclaimed, and bench‐to‐bedside translation appears to be lost in a “valley of death”. Both predicaments, non‐reproducibility and translational roadblocks, are connected: Why should we expect to successfully “trans‐late” results to humans, if already “cis‐lation”—that is, the generalization from one experimental setting to an identical or fairly similar one—often fails?

Timely and comprehensive reporting of clinical trial results builds the backbone of evidence-based medicine and responsible research. The proportion of timely disseminated trial results can inform alternative national and international benchmarking of university medical centers (UMCs).For all German UMCs, we tracked all registered trials completed between 2009 and 2013. The results and an interactive website benchmark German UMCs regarding their performance in result dissemination.We identified and tracked 2,132 clinical trials. For 1,509 trials, one of the German UMCs took the academic lead. Of these 1,509 "lead trials," 39% published their results (mostly via journal publications) in a timely manner (24 months after completion). More than 6 years after study completion, 26% of all eligible lead trials still had not disseminated results.Despite substantial attention from many stakeholders to the topic, there is still a strong delay or even absence of result dissemination for many trials. German UMCs have several opportunities to improve this situation. Further research should evaluate whether and how a transparent benchmarking of UMC performance in result dissemination helps to increase value and reduce waste in medical research.

While high risk of failure is an inherent part of developing innovative therapies, it can be reduced by adherence to evidence-based rigorous research practices. Numerous analyses conducted to date have clearly identified measures that need to be taken to improve research rigor. Supported through the European Union's Innovative Medicines Initiative, the EQIPD consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use. The EQIPD Quality System was designed to be suited to boost innovation by ensuring the generation of robust and reliable preclinical data while being lean, effective and not becoming a burden that could negatively impact the freedom to explore scientific questions. EQIPD defines research quality as the extent to which research data are fit for their intended use. Fitness, in this context, is defined by the stakeholders, who are the scientists directly involved in the research, but also their funders, sponsors, publishers, research tool manufacturers and collaboration partners such as peers in a multi-site research project. The essence of the EQIPD Quality System is the set of 18 core requirements that can be addressed flexibly, according to user-specific needs and following a user-defined trajectory. The EQIPD Quality System proposes guidance on expectations for quality-related measures, defines criteria for adequate processes (i.e., performance standards) and provides examples of how such measures can be developed and implemented. However, it does not prescribe any pre-determined solutions. EQIPD has also developed tools (for optional use) to support users in implementing the system and assessment services for those research units that successfully implement the quality system and seek formal accreditation. Building upon the feedback from users and continuous improvement, a sustainable EQIPD Quality System will ultimately serve the entire community of scientists conducting non-regulated preclinical research, by helping them generate reliable data that are fit for their intended use.

In response to the Hong Kong Principles for assessing researchers, this Formal Comment argues that it is time to take gender and diversity considerations seriously in the pursuit of fostering research integrity; this requires acknowledging and reshaping the influence of research assessment criteria on researcher representation.

The purpose of preclinical research is to inform the development of novel diagnostics or therapeutics, and the results of experiments on animal models of disease often inform the decision to conduct studies in humans. However, a substantial number of clinical trials fail, even when preclinical studies have apparently demonstrated the efficacy of a given intervention. A number of large-scale replication studies are currently trying to identify the factors that influence the robustness of preclinical research. Here, we discuss replications in the context of preclinical research trajectories, and argue that increasing validity should be a priority when selecting experiments to replicate and when performing the replication. We conclude that systematically improving three domains of validity – internal, external and translational – will result in a more efficient allocation of resources, will be more ethical, and will ultimately increase the chances of successful translation.

Connectome analysis of neuroimaging data is a rapidly expanding field to identify disease specific biomarkers. Structural diffusion MRI connectivity has been useful in individuals with radiological features of small vessel disease, such as white matter hyperintensities. Global efficiency, a network metric calculated from the structural connectome, is an excellent predictor of cognitive decline. To dissect the biological underpinning of these changes, animal models are required. We tested whether the structural connectome is altered in a mouse model of vascular cognitive impairment. White matter damage was more pronounced by 6 compared to 3 months. Global efficiency remained intact, but the visual association cortex exhibited increased structural connectivity with other brain regions. Exploratory resting state functional MRI connectivity analysis revealed diminished default mode network activity in the model compared to shams. Further perturbations were observed in a primarily cortical hub and the retrosplenial and visual cortices, and the hippocampus were the most affected nodes. Behavioural deficits were observed in the cued water maze, supporting the suggestion that the visual and spatial memory networks are affected. We demonstrate specific circuitry is rendered vulnerable to vascular stress in the mouse, and the model will be useful to examine pathophysiological mechanisms of small vessel disease.Graphical abstract

Over the last two decades, awareness of the negative repercussions of flaws in the planning, conduct and reporting of preclinical research involving experimental animals has been growing. Several initiatives have set out to increase transparency and internal validity of preclinical studies, mostly publishing expert consensus and experience. While many of the points raised in these various guidelines are identical or similar, they differ in detail and rigour. Most of them focus on reporting, only few of them cover the planning and conduct of studies. The aim of this systematic review is to identify existing experimental design, conduct, analysis and reporting guidelines relating to preclinical animal research. A systematic search in PubMed, Embase and Web of Science retrieved 13 863 unique results. After screening these on title and abstract, 613 papers entered the full-text assessment stage, from which 60 papers were retained. From these, we extracted unique 58 recommendations on the planning, conduct and reporting of preclinical animal studies. Sample size calculations, adequate statistical methods, concealed and randomised allocation of animals to treatment, blinded outcome assessment and recording of animal flow through the experiment were recommended in more than half of the publications. While we consider these recommendations to be valuable, there is a striking lack of experimental evidence on their importance and relative effect on experiments and effect sizes.

Big moves to rebuild the scientific infrastructure are possible. Big moves to rebuild the scientific infrastructure are possible.

Statistically significant findings are more likely to be published than non-significant or null findings, leaving scientists and healthcare personnel to make decisions based on distorted scientific evidence. Continuously expanding ´file drawers’ of unpublished data from well-designed experiments waste resources creates problems for researchers, the scientific community and the public. There is limited awareness of the negative impact that publication bias and selective reporting have on the scientific literature. Alternative publication formats have recently been introduced that make it easier to publish research that is difficult to publish in traditional peer reviewed journals. These include micropublications, data repositories, data journals, preprints, publishing platforms, and journals focusing on null or neutral results. While these alternative formats have the potential to reduce publication bias, many scientists are unaware that these formats exist and don’t know how to use them. Our open source file drawer data liberation effort (fiddle) tool (RRID:SCR_017327 available at: http://s-quest.bihealth.org/fiddle/) is a match-making Shiny app designed to help biomedical researchers to identify the most appropriate publication format for their data. Users can search for a publication format that meets their needs, compare and contrast different publication formats, and find links to publishing platforms. This tool will assist scientists in getting otherwise inaccessible, hidden data out of the file drawer into the scientific community and literature. We briefly highlight essential details that should be included to ensure reporting quality, which will allow others to use and benefit from research published in these new formats.

While high risk of failure is an inherent part of developing innovative therapies, it can be reduced by adherence to evidence-based rigorous research practices. Numerous analyses conducted to date have clearly identified measures that need to be taken to improve research rigor. Supported through the European Union’s Innovative Medicines Initiative, the EQIPD consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use. The EQIPD Quality System was designed to be suited to boost innovation by ensuring the generation of robust and reliable preclinical data while being lean, effective and not becoming a burden that could negatively impact the freedom to explore scientific questions. EQIPD defines research quality as the extent to which research data are fit for their intended use. Fitness, in this context, is defined by the stakeholders, who are the scientists directly involved in the research, but also their funders, sponsors, publishers, research tool manufacturers and collaboration partners such as peers in a multi-site research project. The essence of the EQIPD Quality System is the set of 18 core requirements that can be addressed flexibly, according to user-specific needs and following a user-defined trajectory. The EQIPD Quality System proposes guidance on expectations for quality-related measures, defines criteria for adequate processes (i.e., performance standards) and provides examples of how such measures can be developed and implemented. However, it does not prescribe any pre-determined solutions. EQIPD has also developed tools (for optional use) to support users in implementing the system. Further, EQIPD is preparing training support and assessment services for those research units that successfully implement the quality system and would like to seek formal accreditation. Building upon the feedback from users and continuous improvement, a sustainable EQIPD Quality System will ultimately serve the entire community of scientists conducting non-regulated preclinical research, by helping them generate reliable data that are fit for their intended use.

Para que o conhecimento beneficie a pesquisa e a sociedade, ele deve ser confiável. A pesquisa confiável é robusta, rigorosa e transparente em todos as etapas relativas ao desenho, execução e relato. Apesar desse avanço, a inclusão de considerações relacionadas à confiabilidade, ao rigor e à transparência continua sendo rara. Os Princípios de Hong Kong (HKPs) foram desenvolvidos como parte da 6a Conferência Mundial sobre Integridade em Pesquisa, com foco sobre a necessidade de impulsionar o aprimoramento da pesquisa, garantindo que os pesquisadores sejam explicitamente reconhecidos e recompensados por atitudes que fortalecem a integridade. Apresentamos cinco princípios: práticas de pesquisa responsáveis; relato transparente; ciência aberta (pesquisa aberta); valorização da diversidade de tipos de pesquisa; reconhecimento de todas as contribuições para a pesquisa e atividade acadêmica. Para cada princípio, oferecemos uma justificativa, com exemplos que ilustram onde esses princípios já estão sendo adotados.

For knowledge to benefit research and society, it must be trustworthy. Trustworthy research is robust, rigorous, and transparent at all stages of design, execution, and reporting. Assessment of researchers still rarely includes considerations related to trustworthiness, rigor, and transparency. We have developed the Hong Kong Principles (HKPs) as part of the 6th World Conference on Research Integrity with a specific focus on the need to drive research improvement through ensuring that researchers are explicitly recognized and rewarded for behaviors that strengthen research integrity. We present five principles: responsible research practices; transparent reporting; open science (open research); valuing a diversity of types of research; and recognizing all contributions to research and scholarly activity. For each principle, we provide a rationale for its inclusion and provide examples where these principles are already being adopted., Assessment of researchers still rarely includes considerations related to trustworthiness, rigor, and transparency. This Essay presents the Hong Kong Principles (HKPs), developed as part of the 6th World Conference on Research Integrity, with a specific focus on the need to drive research improvement by ensuring that researchers are explicitly recognized and rewarded for behavior that leads to trustworthy research.

Over the last two decades, awareness of the negative repercussions of flaws in the planning, conduct and reporting of preclinical research involving experimental animals has been growing. Several initiatives have set out to increase transparency and internal validity of preclinical studies, mostly publishing expert consensus and experience. While many of the points raised in these various guidelines are identical or similar, they differ in detail and rigour. Most of them focus on reporting, only few of them cover the planning and conduct of studies. The aim of this systematic review is to identify existing experimental design, conduct, analysis and reporting guidelines relating to preclinical animal research. A systematic search in PubMed, Embase and Web of Science retrieved 13 863 unique results. After screening these on title and abstract, 613 papers entered the full-text assessment stage, from which 60 papers were retained. From these, we extracted unique 58 recommendations on the planning, conduct and reporting of preclinical animal studies. Sample size calculations, adequate statistical methods, concealed and randomised allocation of animals to treatment, blinded outcome assessment and recording of animal flow through the experiment were recommended in more than half of the publications. While we consider these recommendations to be valuable, there is a striking lack of experimental evidence on their importance and relative effect on experiments and effect sizes.

How much can we rely on whether what was reported in a study was actually done? Systematic and independent examination of records, documents and processes through audits are a central element of quality management systems. In the context of current concerns about the robustness and reproducibility of experimental biomedical research audits have been suggested as a remedy a number of times. However, audits are resource intense and time consuming, and due to their very nature may be perceived as inquisition. Consequently, there is very little experience or literature on auditing and assessments in the complex preclinical biomedical research environment. To gain some insight into which audit approaches might best suit biomedical research in academia, in this study we have applied a number of them in a typical academic neuroscience environment consisting of twelve research groups with about 100 researchers, students and technicians, utilizing the full gamut of state of the art methodology. Several types of assessments and internal as well as external audits (including the novel format of a peer audit) were systematically explored by a team of quality management specialists. An experimental design template was developed (and is provided here) that takes into account and mitigates difficulties, risks and systematic errors that may occur during the course of a study. All audits were performed according to a pre-defined workflow developed by us. Outcomes were assessed qualitatively. We asked for feedback from participating employees in every final discussion of an audit and documented this in the audit reports. Based on these reports follow-up audits were improved. We conclude that several realistic options for auditing exist which have the potential to improve preclinical biomedical research in academia, and have listed specific recommendations regarding their benefits and provided practical resources for their implementation (e.g. study design and audit templates, audit workflow).

The reproducibility crisis triggered worldwide initiatives to improve rigor, reproducibility, and transparency in biomedical research. There are many examples of scientists, journals, and funding agencies adopting responsible research practices. The QUEST (Quality-Ethics-Open Science-Translation) Center offers a unique opportunity to examine the role of institutions. The Berlin Institute of Health founded QUEST to increase the likelihood that research conducted at this large academic medical center would be trustworthy, useful for scientists and society, and ethical. QUEST researchers perform “science of science” studies to understand problems with standard practices and develop targeted solutions. The staff work with institutional leadership and local scientists to incentivize and support responsible practices in research, funding, and hiring. Some activities described in this paper focus on the institution, whereas others may benefit the national and international scientific community. Our experience, approaches, and recommendations will be informative for faculty leadership, administrators, and researchers interested in improving scientific practice., This Community Page article describes an ongoing large-scale structured initiative at the Berlin Institute of Health which aims at improving the robustness, reproducibility and transparency of research in an academic biomedical setting.

Brain collateral circulation is an essential compensatory mechanism in response to acute brain ischemia. To study the temporal evolution of brain macro and microcollateral recruitment and their reciprocal interactions in response to different ischemic conditions, we applied a combination of complementary techniques (T2-weighted magnetic resonance imaging [MRI], time of flight [TOF] angiography [MRA], cerebral blood flow [CBF] imaging and histology) in two different mouse models. Hypoperfusion was either induced by permanent bilateral common carotid artery stenosis (BCCAS) or 60-min transient unilateral middle cerebral artery occlusion (MCAO). In both models, collateralization is a very dynamic phenomenon with a global effect affecting both hemispheres. Patency of ipsilateral posterior communicating artery (PcomA) represents the main variable survival mechanism and the main determinant of stroke lesion volume and recovery in MCAO, whereas the promptness of external carotid artery retrograde flow recruitment together with PcomA patency, critically influence survival, brain ischemic lesion volume and retinopathy in BCCAS mice. Finally, different ischemic gradients shape microcollateral density and size.

ZusammenfassungExperimentelle und klinische Studien der letzten Jahre belegen die wichtige Rolle der intestinalen Mikrobiota für die Homöostase des Zentralnervensystems (ZNS), aber auch für die Pathophysiologie von ZNS-Erkrankungen. Dabei spielt u. a. der modifizierende Einfluss der Mikrobiota auf die periphere Immunantwort eine wichtige Rolle, die wiederum einen modulierenden Effekt auf die Neuroinflammation im ZNS haben kann. Hier fassen wir die verfügbaren experimentellen und klinischen Daten zur Bedeutung des Mikrobioms beim Schlaganfall zusammen: 1) Schlaganfall verändert die intestinale Mikrobiota; 2) Mikrobiota kann die Prognose nach Schlaganfall beeinflussen; 3) Mikrobiota spielt eine wichtige Rolle in der Pathogenese von (Risikofaktoren) des Schlaganfalls. Aufgrund der vorliegenden klinischen und experimentellen Ergebnisse stehen für die nächsten Jahre klinisch relevante Entdeckungen zur Bedeutung der Darm-Mikrobiota beim Schlaganfall an.

Recent studies have provided evidence for the role of the gut microbiota in the homeostasis of the central nervous system (CNS) and in the pathophysiology of its disorders, e. g. by regulation of the peripheral immune response. In this article, we discuss the importance of the gut microbiota in stroke by providing a summary of available clinical and experimental data suggesting that 1) stroke changes the gut microbiome, 2) microbiota modulates stroke outcome and 3) microbiota plays an important role in the pathogenesis of stroke (risk factors). Currently available clinical and experimental evidence suggests an important role of gut microbiota in stroke and promises clinically relevant discoveries within coming years.

Folates are B-vitamins that are vital for normal brain function. Deficiencies in folates either genetic (methylenetetrahydrofolate reductase, MTHFR) or dietary intake of folic acid result in elevated levels of homocysteine. Clinical studies have shown that elevated levels of homocysteine (Hcy) may be associated with the development of dementia, however this link remains unclear. The purpose of this study was to evaluate the impact of increased Hcy levels on a mouse model of vascular cognitive impairment (VCI) produced by chronic hypoperfusion. Male and female Mthfr+/+ and Mthfr+/- mice were placed on either control (CD) or folic acid deficient (FADD) diets after which all animals underwent microcoil implantation around each common carotid artery or a sham procedure. Post-operatively animals were tested on the Morris water maze (MWM), y-maze, and rotarod. Animals had no motor impairments on the rotarod, y-maze, and could learn the location of the platform on the MWM. However, on day 8 of testing of MWM testing during the probe trial, Mthfr+/- FADD microcoil mice spent significantly less time in the target quadrant when compared to Mthfr+/- CD sham mice, suggesting impaired reference memory. All FADD mice had elevated levels of plasma homocysteine. MRI analysis revealed arterial remodeling was present in Mthfr+/- microcoil mice not Mthfr+/+ mice. Acetylcholine and related metabolites were reduced in cortical tissue because of microcoil implantation and elevated levels of homocysteine. Deficiencies in folate metabolism resulting in increased Hcy levels yield a metabolic profile that increases susceptibility to neurodegeneration in a mouse model of VCI.

ObjectiveTo explore the accuracy of combined neurology expert forecasts in predicting primary endpoints for trials.MethodsWe identified one major randomized trial each in stroke, multiple sclerosis (MS), and amyotrophic lateral sclerosis (ALS) that was closing within 6 months. After recruiting a sample of neurology experts for each disease, we elicited forecasts for the primary endpoint outcomes in the trial placebo and treatment arms. Our main outcome was the accuracy of averaged predictions, measured using ordered Brier scores. Scores were compared against an algorithm that offered noncommittal predictions.ResultsSeventy-one neurology experts participated. Combined forecasts of experts were less accurate than a noncommittal prediction algorithm for the stroke trial (pooled Brier score = 0.340, 95% subjective probability interval [sPI] 0.340 to 0.340 vs 0.185 for the uninformed prediction), and approximately as accurate for the MS study (pooled Brier score = 0.107, 95% confidence interval [CI] 0.081 to 0.133 vs 0.098 for the noncommittal prediction) and the ALS study (pooled Brier score = 0.090, 95% CI 0.081 to 0.185 vs 0.090). The 95% sPIs of individual predictions contained actual trial outcomes among 44% of experts. Only 18% showed prediction skill exceeding the noncommittal prediction. Independent experts and coinvestigators achieved similar levels of accuracy.ConclusionIn this first-of-kind exploratory study, averaged expert judgments rarely outperformed noncommittal forecasts. However, experts at least anticipated the possibility of effects observed in trials. Our findings, if replicated in different trial samples, caution against the reliance on simple approaches for combining expert opinion in making research and policy decisions.

Supplemental Digital Content is available in the text., Background and Purpose— Preclinical research using animals often informs clinical trials. However, its value is dependent on its scientific validity and reproducibility, which are, in turn, dependent on rigorous study design and reporting. In 2011, Stroke introduced a Basic Science Checklist to enhance the reporting and methodology of its preclinical studies. Except for Nature and Science journals, few others have implemented similar initiatives. We sought to estimate the impact of these journal interventions on the quality of their published reports. Methods— All articles published in Stroke, Nature Medicine, and Science Translational Medicine over 9 to 18 years and in 2 control journals without analogous interventions over a corresponding 11.5 years were reviewed to identify reports of experiments in nonhuman mammals with proposed clinical relevance. The effect of journal interventions on the reporting and use of key study design elements was estimated via interrupted time-series analyses. Results— Of 33 009 articles screened, 4162 studies met inclusion criteria. In the 3.5 to 12 years preceding each journal’s intervention, the proportions of studies reporting and using key study design elements were stable except for blinding in Stroke and randomization in Science Translational Medicine, which were both increasing. Post-intervention, abrupt and often marked increases were seen in the reporting of randomization status (level change: +17% to +44%, P≤0.005), blinding (level change: +20% to +40%, P≤0.008), and sample size estimation (level change: 0% to +40%, P≤0.002 in 2 journals). Significant but more modest improvements in the use of these study design elements were also observed. These improvements were not seen in control journals. Conclusions— Journal interventions such as Stroke’s author submission checklist can meaningfully improve the quality of published preclinical research and should be considered to enhance study transparency and design. However, such interventions are alone insufficient to fully address widespread shortcomings in preclinical research practices.

Confirmation through competent replication is a founding principle of modern science. However, biomedical researchers are rewarded for innovation, and not for confirmation, and confirmatory research is often stigmatized as unoriginal and as a consequence faces barriers to publication. As a result, the current biomedical literature is dominated by exploration, which to complicate matters further is often disguised as confirmation. Only recently scientists and the public have begun to realize that high-profile research results in biomedicine can often not be replicated. Consequently, confirmation has become central stage in the quest to safeguard the robustness of research findings. Research which is pushing the boundaries of or challenges what is currently known must necessarily result in a plethora of false positive results. Thus, since discovery, the driving force of scientific progress, is unavoidably linked to high false positive rates and cannot support confirmatory inference, dedicated confirmatory investigation is needed for pivotal results. In this chapter I will argue that the tension between the two modes of research, exploration and confirmation, can be resolved if we conceptually and practically separate them. I will discuss the idiosyncrasies of exploratory and confirmatory studies, with a focus on the specific features of their design, analysis, and interpretation.

Objective Maternal autoantibodies are a risk factor for impaired brain development in offspring. Antibodies (ABs) against the NR1 (GluN1) subunit of the N-methyl-d-aspartate receptor (NMDAR) are among the most frequently diagnosed anti-neuronal surface ABs, yet little is known about effects on fetal development during pregnancy. Methods We established a murine model of in utero exposure to human recombinant NR1 and isotype-matched nonreactive control ABs. Pregnant C57BL/6J mice were intraperitoneally injected on embryonic days 13 and 17 each with 240μg of human monoclonal ABs. Offspring were investigated for acute and chronic effects on NMDAR function, brain development, and behavior. Results Transferred NR1 ABs enriched in the fetus and bound to synaptic structures in the fetal brain. Density of NMDAR was considerably reduced (up to -49.2%) and electrophysiological properties were altered, reflected by decreased amplitudes of spontaneous excitatory postsynaptic currents in young neonates (-34.4%). NR1 AB-treated animals displayed increased early postnatal mortality (+27.2%), impaired neurodevelopmental reflexes, altered blood pH, and reduced bodyweight. During adolescence and adulthood, animals showed hyperactivity (+27.8% median activity over 14 days), lower anxiety, and impaired sensorimotor gating. NR1 ABs caused long-lasting neuropathological effects also in aged mice (10 months), such as reduced volumes of cerebellum, midbrain, and brainstem. Interpretation The data collectively support a model in which asymptomatic mothers can harbor low-level pathogenic human NR1 ABs that are diaplacentally transferred, causing neurotoxic effects on neonatal development. Thus, AB-mediated network changes may represent a potentially treatable neurodevelopmental congenital brain disorder contributing to lifelong neuropsychiatric morbidity in affected children. ANN NEUROL 2019;86:656-670.

Over the last two decades, awareness of the negative repercussions of flaws in the planning, conduct and reporting of preclinical research involving experimental animals has been growing. Several initiatives have set out to increase transparency and internal validity of preclinical studies, mostly publishing expert consensus and experience. While many of the points raised in these various guidelines are identical or similar, they differ in detail and rigour. Most of them focus on reporting, only few of them cover the planning and conduct of studies. The aim of this systematic review is to identify existing experimental design, conduct, analysis and reporting guidelines relating to preclinical animal research. A systematic search in PubMed, Embase and Web of Science retrieved 13 863 unique results. After screening these on title and abstract, 613 papers entered the full-text assessment stage, from which 60 papers were retained. From these, we extracted unique 58 recommendations on the planning, conduct and reporting of preclinical animal studies. Sample size calculations, adequate statistical methods, concealed and randomised allocation of animals to treatment, blinded outcome assessment and recording of animal flow through the experiment were recommended in more than half of the publications. While we consider these recommendations to be valuable, there is a striking lack of experimental evidence on their importance and relative effect on experiments and effect sizes.

Reproducible science requires transparent reporting. The ARRIVE guidelines were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved.Here we introduce ARRIVE 2019. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise the items and split the guidelines into two sets, the ARRIVE Essential 10, which constitute the minimum requirement, and the Recommended Set, which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers to verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document that serves 1) to explain the rationale behind each item in the guidelines, 2) to clarify key concepts and 3) to provide illustrative examples. We aim through these changes to help ensure that researchers, reviewers and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

Improving the reproducibility of biomedical research is a major challenge. Transparent and accurate reporting are vital to this process; it allows readers to assess the reliability of the findings, and repeat or build upon the work of other researchers. The NC3Rs developed the ARRIVE guidelines in 2010 to help authors and journals identify the minimum information necessary to report in publications describing in vivo experiments.Despite widespread endorsement by the scientific community, the impact of the ARRIVE guidelines on the transparency of reporting in animal research publications has been limited. We have revised the ARRIVE guidelines to update them and facilitate their use in practice. The revised guidelines are published alongside this paper. This Explanation and Elaboration document was developed as part of the revision. It provides further information about each of the 21 items in ARRIVE 2019, including the rationale and supporting evidence for their inclusion in the guidelines, elaboration of details to report, and examples of good reporting from the published literature.

ObjectiveTo determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation.MethodsWe investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 preclinical SCI studies including 9,535 animals were analyzed with meta-regression to assess the effect of various study characteristics and the quality of neurologic recovery.ResultsOverall, the included interventions reported a neurobehavioral outcome improvement of 26.3% (95% confidence interval 24.3–28.4). Response to treatment was dependent on experimental modeling paradigms (neurobehavioral score, site of injury, and animal species). Applying multiple outcome measures was consistently associated with smaller effect sizes compared with studies applying only 1 outcome measure. More than half of the studies (51.2%) did not report blinded assessment, constituting a likely source of evaluation bias, with an overstated effect size of 7.2%. Assessment of publication bias, which extrapolates to identify likely missing data, suggested that between 2% and 41% of experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy, reducing the effect sizes by between 0.9% and 14.3%.ConclusionsWe provide empirical evidence of prevalent bias in the design and reporting of experimental SCI studies, resulting in overestimation of the effectiveness. Bias compromises the internal validity and jeopardizes the successful translation of SCI therapies from the bench to bedside.

The current, widely established 3R framework for the ethical use of animals in research consists of three guiding principles, that is, Replacement, Reduction and Refinement, all aiming to safeguard the overarching ethical principle of animal welfare. However, animal welfare alone does not suffice to make animal research ethical if the research does not have sufficient scientific value. The scientific value of animal studies strongly decreases if they are not sufficiently robust, if their questions have already been sufficiently addressed or if the results are selectively reported. Against this background, we argue that three guiding principles are missing, that is, Robustness, Registration and Reporting, all of which aim to safeguard and increase the scientific value of animal research. To establish a new 6R framework, we need a multistakeholder discourse to conceptualise the specific requirements of robustness, registration and reporting and to clarify responsibilities, competencies and legislation for auditing 6R compliance.