Search

ObjectiveWe sought to apply natural language processing to the task of automatic risk of bias assessment in preclinical literature, which could speed the process of systematic review, provide information to guide research improvement activity, and support translation from preclinical to clinical research.Materials and MethodsWe use 7,840 full-text publications describing animal experiments with yes/no annotations for five risk of bias items. We implement a series of models including baselines (support vector machine, logistic regression, random forest), neural models (convolutional neural network, recurrent neural network with attention, hierarchical neural network) and models using BERT with two strategies (document chunk pooling and sentence extraction). We tune hyperparameters to obtain the highest F1 scores for each risk of bias item on the validation set and compare evaluation results on the test set to our previous regular expression approach.ResultsThe F1 scores of best models on test set are 82.0% for random allocation, 81.6% for blinded assessment of outcome, 82.6% for conflict of interests, 91.4% for compliance with animal welfare regulations and 46.6% for reporting animals excluded from analysis. Our models significantly outperform regular expressions for four risk of bias items.ConclusionFor random allocation, blinded assessment of outcome, conflict of interests and animal exclusions, neural models achieve good performance, and for animal welfare regulations, BERT model with sentence extraction strategy works better.

Background Natural language processing could assist multiple tasks in systematic reviews to reduce workflow, including the extraction of PICO elements such as study populations, interventions, comparators and outcomes. The PICO framework provides a basis for the retrieval and selection for inclusion of evidence relevant to a specific systematic review question, and automatic approaches to PICO extraction have been developed particularly for reviews of clinical trial findings. Considering the difference between preclinical animal studies and clinical trials, developing separate approaches is necessary. Facilitating preclinical systematic reviews will inform the translation from preclinical to clinical research. Methods We randomly selected 400 abstracts from the PubMed Central Open Access database which described in vivo animal research and manually annotated these with PICO phrases for Species, Strain, methods of Induction of disease model, Intervention, Comparator and Outcome. We developed a two-stage workflow for preclinical PICO extraction. Firstly we fine-tuned BERT with different pre-trained modules for PICO sentence classification. Then, after removing the text irrelevant to PICO features, we explored LSTM-, CRF- and BERT-based models for PICO entity recognition. We also explored a self-training approach because of the small training corpus. Results For PICO sentence classification, BERT models using all pre-trained modules achieved an F1 score of over 80%, and models pre-trained on PubMed abstracts achieved the highest F1 of 85%. For PICO entity recognition, fine-tuning BERT pre-trained on PubMed abstracts achieved an overall F1 of 71% and satisfactory F1 for Species (98%), Strain (70%), Intervention (70%) and Outcome (67%). The score of Induction and Comparator is less satisfactory, but F1 of Comparator can be improved to 50% by applying self-training. Conclusions Our study indicates that of the approaches tested, BERT pre-trained on PubMed abstracts is the best for both PICO sentence classification and PICO entity recognition in the preclinical abstracts. Self-training yields better performance for identifying comparators and strains.

IntroductionMotor neuron disease (MND) is a rapidly fatal neurodegenerative disease. Despite decades of research and clinical trials there remains no cure and only one globally approved drug, riluzole, which prolongs survival by 2–3 months. Recent improved mechanistic understanding of MND heralds a new translational era with many potential targets being identified that are ripe for clinical trials. Motor Neuron Disease Systematic Multi-Arm Adaptive Randomised Trial (MND-SMART) aims to evaluate the efficacy of drugs efficiently and definitively in a multi-arm, multi-stage, adaptive trial. The first two drugs selected for evaluation in MND-SMART are trazodone and memantine.Methods and analysisInitially, up to 531 participants (177/arm) will be randomised 1:1:1 to oral liquid trazodone, memantine and placebo. The coprimary outcome measures are the Amyotrophic Lateral Sclerosis Functional Rating Scale Revised (ALSFRS-R) and survival. Comparisons will be conducted in four stages. The decision to continue randomising to arms after each stage will be made by the Trial Steering Committee who receive recommendations from the Independent Data Monitoring Committee. The primary analysis of ALSFRS-R will be conducted when 150 participants/arm, excluding long survivors, have completed 18 months of treatment; if positive the survival effect will be inferentially analysed when 113 deaths have been observed in the placebo group. The trial design ensures that other promising drugs can be added for evaluation in planned trial adaptations. Using this novel trial design reduces time, cost and number of participants required to definitively (phase III) evaluate drugs and reduces exposure of participants to potentially ineffective treatments.Ethics and disseminationMND-SMART was approved by the West of Scotland Research Ethics Committee on 2 October 2019. (REC reference: 19/WS/0123) Results of the study will be submitted for publication in a peer-reviewed journal and a summary provided to participants.Trial registration numbersEuropean Clinical Trials Registry (2019-000099-41); NCT04302870.

Based on previous case reports and disease-based cohorts, a minority of patients with cerebral small vessel disease (cSVD) have a monogenic cause, with many also manifesting extra-cerebral phenotypes. We investigated the frequency, penetrance, and phenotype associations of rare variants in cSVD genes in UK Biobank (UKB), a large population-based study.We used a systematic review of previous literature and ClinVar to identify putative pathogenic rare variants in CTSA, TREX1, HTRA1, COL4A1/2. We mapped phenotypes previously attributed to these variants (phenotypes-of-interest) to disease coding systems used in UKB’s linked health data from UK hospital admissions, death records and primary care. Among 199,313 exome-sequenced UKB participants, we assessed: the proportion of participants carrying ≥1 variant(s); phenotype-of-interest penetrance; and the association between variant carrier status and phenotypes-of-interest using a binary (any phenotype present/absent) and phenotype burden (linear score of the number of phenotypes a participant possessed) approach.Among UKB participants, 0.5% had ≥1 variant(s) in studied genes. Using hospital admission and death records, 4-20% of variant carriers per gene had an associated phenotype. This increased to 7-55% when including primary care records. Only COL4A1 variant carrier-status was significantly associated with having ≥1 phenotype-of-interest and a higher phenotype score (OR=1.29, p=0.006).While putative pathogenic rare variants in monogenic cSVD genes occur in 1:200 people in the UKB population, only around half of variant carriers have a relevant disease phenotype recorded in their linked health data. We could not replicate most previously reported gene-phenotype associations, suggesting lower penetrance rates, overestimated pathogenicity and/or limited statistical power.

Background Time-to-event data is frequently reported in both clinical and preclinical research spheres. Systematic review and meta-analysis is a tool that can help to identify pitfalls in preclinical research conduct and reporting that can help to improve translational efficacy. However, pooling of studies using hazard ratios (HRs) is cumbersome especially in preclinical meta-analyses including large numbers of small studies. Median survival is a much simpler metric although because of some limitations, which may not apply to preclinical data, it is generally not used in survival meta-analysis. We aimed to appraise its performance when compared with hazard ratio-based meta-analysis when pooling large numbers of small, imprecise studies. Methods We simulated a survival dataset with features representative of a typical preclinical survival meta-analysis, including with influence of a treatment and a number of covariates. We calculated individual patient data-based hazard ratios and median survival ratios (MSRs), comparing the summary statistics directly and their performance at random-effects meta-analysis. Finally, we compared their sensitivity to detect associations between treatment and influential covariates at meta-regression. Results There was an imperfect correlation between MSR and HR, although the opposing direction of treatment effects between summary statistics appeared not to be a major issue. Precision was more conservative for HR than MSR, meaning that estimates of heterogeneity were lower. There was a slight sensitivity advantage for MSR at meta-analysis and meta-regression, although power was low in all circumstances. Conclusions We believe we have validated MSR as a summary statistic for use in a meta-analysis of small, imprecise experimental survival studies—helping to increase confidence and efficiency in future reviews in this area. While assessment of study precision and therefore weighting is less reliable, MSR appears to perform favourably during meta-analysis. Sensitivity of meta-regression was low for this set of parameters, so pooling of treatments to increase sample size may be required to ensure confidence in preclinical survival meta-regressions.

Researchers, institutions, funders, and publishers are considering how to improve research culture and quality, but no single part of the research ecosystem can effect change on its own. The UK Reproducibility Network (UKRN) was established to facilitate the necessary coordination. Its experience can inform the establishment of like-minded networks around the world to drive positive change.

BackgroundHeterogeneity of results of exact same research experiments oppose a significant socio-economic burden. In vitro research presents the early step of basic science and drug development projects. Insufficient methodological reporting is likely to be one of the contributors to results heterogeneity, however, little knowledge on reporting habits of in vitro cancer research and their effects on results reproducibility is available. Glioblastoma is a form of brain cancer with largely unmet clinical need.MethodsHere we use systematic review to describe reporting practices in in vitro glioblastoma research using the U87-MG cell line and perform multilevel random-effects meta-analysis followed by meta-regression to explore sources of heterogeneity within that literature, and any associations between reporting characteristics and reported findings.ResultsIn 137 identified articles, the overall methodological reporting is disappointing, e.g., the control type, mediums glucose level and cell density are reported in only 36.5, 21.2 and 16.8 percent of the articles, respectively. After adjustments for different drug concentrations and treatment durations, a three-level meta-analysis proves meaningful results heterogeneity across the studies (I2 = 70.1%).ConclusionsOur results further support the ongoing efforts of establishing consensus reporting practices to elevate durability of results. By doing so, we hope that this work will raise awareness of how stricter reporting may help to improve the frequency of successful translation of preclinical results into human application, not only in neuro-oncology.FundingWe received no specific funding for this project.

Background Meta-analysis of preclinical data is used to evaluate the consistency of findings and to inform the design and conduct of future studies. Unlike clinical meta-analysis, preclinical data often involve many heterogeneous studies reporting outcomes from a small number of animals. Here, we review the methodological challenges in preclinical meta-analysis in estimating and explaining heterogeneity in treatment effects. Methods Assuming aggregate-level data, we focus on two topics: (1) estimation of heterogeneity using commonly used methods in preclinical meta-analysis: method of moments (DerSimonian and Laird; DL), maximum likelihood (restricted maximum likelihood; REML) and Bayesian approach; (2) comparison of univariate versus multivariable meta-regression for adjusting estimated treatment effects for heterogeneity. Using data from a systematic review on the efficacy of interleukin-1 receptor antagonist in animals with stroke, we compare these methods, and explore the impact of multiple covariates on the treatment effects. Results We observed that the three methods for estimating heterogeneity yielded similar estimates for the overall effect, but different estimates for between-study variability. The proportion of heterogeneity explained by a covariate is estimated larger using REML and the Bayesian method as compared with DL. Multivariable meta-regression explains more heterogeneity than univariate meta-regression. Conclusions Our findings highlight the importance of careful selection of the estimation method and the use of multivariable meta-regression to explain heterogeneity. There was no difference between REML and the Bayesian method and both methods are recommended over DL. Multiple meta-regression is worthwhile to explain heterogeneity by more than one variable, reducing more variability than any univariate models and increasing the explained proportion of heterogeneity.

Background The ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines are widely endorsed but compliance is limited. We sought to determine whether journal-requested completion of an ARRIVE checklist improves full compliance with the guidelines. Methods In a randomised controlled trial, manuscripts reporting in vivo animal research submitted to PLOS ONE (March–June 2015) were randomly allocated to either requested completion of an ARRIVE checklist or current standard practice. Authors, academic editors, and peer reviewers were blinded to group allocation. Trained reviewers performed outcome adjudication in duplicate by assessing manuscripts against an operationalised version of the ARRIVE guidelines that consists 108 items. Our primary outcome was the between-group differences in the proportion of manuscripts meeting all ARRIVE guideline checklist subitems. Results We randomised 1689 manuscripts (control: n = 844, intervention: n = 845), of which 1269 were sent for peer review and 762 (control: n = 340; intervention: n = 332) accepted for publication. No manuscript in either group achieved full compliance with the ARRIVE checklist. Details of animal husbandry (ARRIVE subitem 9b) was the only subitem to show improvements in reporting, with the proportion of compliant manuscripts rising from 52.1 to 74.1% (X 2 = 34.0, df = 1, p = 2.1 × 10−7) in the control and intervention groups, respectively. Conclusions These results suggest that altering the editorial process to include requests for a completed ARRIVE checklist is not enough to improve compliance with the ARRIVE guidelines. Other approaches, such as more stringent editorial policies or a targeted approach on key quality items, may promote improvements in reporting.

Background Here, we outline a method of applying existing machine learning (ML) approaches to aid citation screening in an on-going broad and shallow systematic review of preclinical animal studies. The aim is to achieve a high-performing algorithm comparable to human screening that can reduce human resources required for carrying out this step of a systematic review. Methods We applied ML approaches to a broad systematic review of animal models of depression at the citation screening stage. We tested two independently developed ML approaches which used different classification models and feature sets. We recorded the performance of the ML approaches on an unseen validation set of papers using sensitivity, specificity and accuracy. We aimed to achieve 95% sensitivity and to maximise specificity. The classification model providing the most accurate predictions was applied to the remaining unseen records in the dataset and will be used in the next stage of the preclinical biomedical sciences systematic review. We used a cross-validation technique to assign ML inclusion likelihood scores to the human screened records, to identify potential errors made during the human screening process (error analysis). Results ML approaches reached 98.7% sensitivity based on learning from a training set of 5749 records, with an inclusion prevalence of 13.2%. The highest level of specificity reached was 86%. Performance was assessed on an independent validation dataset. Human errors in the training and validation sets were successfully identified using the assigned inclusion likelihood from the ML model to highlight discrepancies. Training the ML algorithm on the corrected dataset improved the specificity of the algorithm without compromising sensitivity. Error analysis correction leads to a 3% improvement in sensitivity and specificity, which increases precision and accuracy of the ML algorithm. Conclusions This work has confirmed the performance and application of ML algorithms for screening in systematic reviews of preclinical animal studies. It has highlighted the novel use of ML algorithms to identify human error. This needs to be confirmed in other reviews with different inclusion prevalence levels, but represents a promising approach to integrating human decisions and automation in systematic review methodology. Electronic supplementary material The online version of this article (10.1186/s13643-019-0942-7) contains supplementary material, which is available to authorized users.

Background Hospital-based studies have reported variable associations between outcome after spontaneous intracerebral hemorrhage and peri-hematomal edema volume. Aims In a community-based study, we aimed to investigate the existence, strength, direction, and independence of associations between intracerebral hemorrhage and peri-hematomal edema volumes on diagnostic brain CT and one-year functional outcome and long-term survival. Methods We identified all adults, resident in Lothian, diagnosed with first-ever, symptomatic spontaneous intracerebral hemorrhage between June 2010 and May 2013 in a community-based, prospective inception cohort study. We defined regions of interest manually and used a semi-automated approach to measure intracerebral hemorrhage volume, peri-hematomal edema volume, and the sum of these measurements (total lesion volume) on first diagnostic brain CT performed at ≤3 days after symptom onset. The primary outcome was death or dependence (scores 3–6 on the modified Rankin Scale) at one-year after intracerebral hemorrhage. Results Two hundred ninety-two (85%) of 342 patients (median age 77.5 y, IQR 68–83, 186 (54%) female, median time from onset to CT 6.5 h (IQR 2.9–21.7)) were dead or dependent one year after intracerebral hemorrhage. Peri-hematomal edema and intracerebral hemorrhage volumes were colinear ( R2 = 0.77). In models using both intracerebral hemorrhage and peri-hematomal edema, 10 mL increments in intracerebral hemorrhage (adjusted odds ratio (aOR) 1.72 (95% CI 1.08–2.87); p = 0.029) but not peri-hematomal edema volume (aOR 0.92 (0.63–1.45); p = 0.69) were independently associated with one-year death or dependence. 10 mL increments in total lesion volume were independently associated with one-year death or dependence (aOR 1.24 (1.11–1.42); p = 0.0004). Conclusion Total volume of intracerebral hemorrhage and peri-hematomal edema, and intracerebral hemorrhage volume alone on diagnostic brain CT, undertaken at three days or sooner, are independently associated with death or dependence one-year after intracerebral hemorrhage, but peri-hematomal edema volume is not. Data access statement Anonymized summary data may be requested from the corresponding author.

While high risk of failure is an inherent part of developing innovative therapies, it can be reduced by adherence to evidence-based rigorous research practices. Numerous analyses conducted to date have clearly identified measures that need to be taken to improve research rigor. Supported through the European Union's Innovative Medicines Initiative, the EQIPD consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use. The EQIPD Quality System was designed to be suited to boost innovation by ensuring the generation of robust and reliable preclinical data while being lean, effective and not becoming a burden that could negatively impact the freedom to explore scientific questions. EQIPD defines research quality as the extent to which research data are fit for their intended use. Fitness, in this context, is defined by the stakeholders, who are the scientists directly involved in the research, but also their funders, sponsors, publishers, research tool manufacturers and collaboration partners such as peers in a multi-site research project. The essence of the EQIPD Quality System is the set of 18 core requirements that can be addressed flexibly, according to user-specific needs and following a user-defined trajectory. The EQIPD Quality System proposes guidance on expectations for quality-related measures, defines criteria for adequate processes (i.e., performance standards) and provides examples of how such measures can be developed and implemented. However, it does not prescribe any pre-determined solutions. EQIPD has also developed tools (for optional use) to support users in implementing the system and assessment services for those research units that successfully implement the quality system and seek formal accreditation. Building upon the feedback from users and continuous improvement, a sustainable EQIPD Quality System will ultimately serve the entire community of scientists conducting non-regulated preclinical research, by helping them generate reliable data that are fit for their intended use.

The replicability of research results has been a cause of increasing concern to the scientific community. The long-held belief that experimental standardization begets replicability has also been recently challenged, with the observation that the reduction of variability within studies can lead to idiosyncratic, lab-specific results that cannot be replicated. An alternative approach is to, instead, deliberately introduce heterogeneity, known as “heterogenization” of experimental design. Here, we explore a novel perspective in the heterogenization program in a meta-analysis of variability in observed phenotypic outcomes in both control and experimental animal models of ischemic stroke. First, by quantifying interindividual variability across control groups, we illustrate that the amount of heterogeneity in disease state (infarct volume) differs according to methodological approach, for example, in disease induction methods and disease models. We argue that such methods may improve replicability by creating diverse and representative distribution of baseline disease state in the reference group, against which treatment efficacy is assessed. Second, we illustrate how meta-analysis can be used to simultaneously assess efficacy and stability (i.e., mean effect and among-individual variability). We identify treatments that have efficacy and are generalizable to the population level (i.e., low interindividual variability), as well as those where there is high interindividual variability in response; for these, latter treatments translation to a clinical setting may require nuance. We argue that by embracing rather than seeking to minimize variability in phenotypic outcomes, we can motivate the shift toward heterogenization and improve both the replicability and generalizability of preclinical research., A meta-analysis study of the variability in phenotypic outcomes in both control and experimental animal models of ischaemic stroke provides novel perspectives in which heterogeneity can be embraced to improve the reproducibility and translation of preclinical studies.

BackgroundResearchers who perform systematic searches across multiple databases often identify duplicate publications. Identifying such duplicates (“deduplication”) can be extremely time-consuming, but failure to remove these citations can, in the worst instance, lead to the wrongful inclusion of duplicate data. Many existing tools are not sensitive enough, lack interoperability with other tools, are not freely accessible, or are difficult to use without programming knowledge. Here, we report the performance of our Automated Systematic Search Deduplicator (ASySD), a novel tool to perform automated deduplication of systematic searches for biomedical reviews.MethodsWe evaluated ASySD’s performance on 5 unseen biomedical systematic search datasets of various sizes (1,845 – 79,880 citations), which had been deduplicated by human reviewers. We compared the performance of ASySD with Endnote’s automated deduplication option and with the Systematic Review Accelerator Deduplication Module (SRA-DM).ResultsASySD identified more duplicates than either SRA-DM or Endnote, with a sensitivity in different datasets of 0.95 to 0.99. The false-positive rate was comparable to human performance, with a specificity of 0.94-0.99. The tool took less than 1 hour to deduplicate all datasets.ConclusionsFor duplicate removal in biomedical systematic reviews, ASySD is a highly sensitive, reliable, and time-saving tool. It is open source and freely available online as both an R package and a user-friendly web application.

Transparency in reporting benefits scientific communication on many levels. While specific needs and expectations vary across fields, the effective interpretation and use of research findings relies on the availability of core information about research materials, study design, data, and experimental and analytical methods. For preclinical research, transparency in reporting is a key focus in response to concerns of replication failure. The inconsistent reporting of key elements of experimental and analytical design, alongside ambiguous description of reagents and lack of access to underlying data and code, has been shown to impair replication (1) and raise doubt about the robustness of results (2, 3). In response to early concerns about replication of published results, funders, publishers, and other stakeholders have called for improvements in reporting transparency (4⇓⇓–7). Several initiatives ensued, including journal policies and joint efforts by journals, funders, and other stakeholders (8⇓–10). One of these initiatives, the Transparency and Openness Promotion (TOP) guidelines (11), outlines a policy framework at the journal level that over 1,000 journals and publishers have adopted. The National Academies have focused on reproducibility and replicability* challenges through several recent initiatives leading to consensus reports, including Reproducibility and Replicability in Science (12), Open Science by Design: Realizing a Vision for 21st Century Research (13), and Fostering Integrity in Research (14). Each of these reports concludes that lack of reporting transparency is one factor which contributes to these systemic problems. Building on these findings, the National Academies convened a public workshop in September 2019 titled “Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting.” The workshop was designed to discuss the current state of transparency in reporting biomedical research and to explore the possibility of improving the harmonization of guidelines across journals and funding agencies. During this workshop, we provided … [↵][1]1To whom correspondence may be addressed: Email: Malcolm.Macleod{at}ed.ac.uk (correspondence regarding the development and evaluation of the guideline) or mellor.david{at}gmail.com (for further information about implementation and stewardship). [1]: #xref-corresp-1-1

Introduction The initiation and conduct of randomised clinical trials are complicated by multiple barriers, including delays in obtaining regulatory approvals. Quantitative data on the extent of the delays due to national or local review in randomised clinical trials is scarce. Materials and methods We assessed the times needed to obtain regulatory approval and to initiate a trial site for an academic, EU-funded, phase III, randomised clinical trial of pharmacological prevention of complications in patients with acute stroke in over 80 sites in nine European countries. The primary outcome was the time from the first submission to a regulatory authority to initiation of a trial site. Secondary outcomes included time needed to complete each individual preparatory requirement and the number of patients recruited by each site in the first 6 and 12 months. Results The median time from the first submission to a regulatory authority to initiation of a trial site was 784 days (IQR: 586–1102). The single most time-consuming step was the conclusion of a clinical trial agreement between the national coordinator and the trial site, which took a median of 194 days (IQR: 93–293). A longer time to site initiation was associated with a lower patient recruitment rate in the first six months after initiation (B = –0.002; p = 0.02). Discussion Conclusion In this EU-funded clinical trial, approximately 26 months were needed to initiate a trial site for patient recruitment. The conclusion of a contract with a trial site was the most time-consuming activity. To simplify and speed up the process, we suggest that the level of detail of contracts for academic trials should be proportional to the risks and commercial interests of these trials.

Preclinical research is a vital step in the drug discovery pipeline and more generally in helping to better understand human disease aetiology and its management. Systematic reviews (SRs) can be powerful in summarising and appraising this evidence concerning a specific research question, to highlight areas of improvements, areas for further research and areas where evidence may be sufficient to take forward to other research domains, for instance clinical trial. Guidance and tools for preclinical research synthesis remain limited despite their clear utility. We aimed to create an online end-to-end platform primarily for conducting SRs of preclinical studies, that was flexible enough to support a wide variety of experimental designs, was adaptable to different research questions, would allow users to adopt emerging automated tools and support them during their review process using best practice. In this article, we introduce the Systematic Review Facility (https://syrf.org.uk), which was launched in 2016 and designed to support primarily preclinical SRs from small independent projects to large, crowdsourced projects. We discuss the architecture of the app and its features, including the opportunity to collaborate easily, to efficiently manage projects, to screen and annotate studies for important features (metadata), to extract outcome data into a secure database, and tailor these steps to each project. We introduce how we are working to leverage the use of automation tools and allow the integration of these services to accelerate and automate steps in the systematic review workflow.

Amyotrophic lateral sclerosis is a progressive and devastating neurodegenerative disease. Despite decades of clinical trials, effective disease-modifying drugs remain scarce. To understand the challenges of trial design and delivery, we performed a systematic review of Phase II, Phase II/III and Phase III amyotrophic lateral sclerosis clinical drug trials on trial registries and PubMed between 2008 and 2019. We identified 125 trials, investigating 76 drugs and recruiting more than 15 000 people with amyotrophic lateral sclerosis. About 90% of trials used traditional fixed designs. The limitations in understanding of disease biology, outcome measures, resources and barriers to trial participation in a rapidly progressive, disabling and heterogenous disease hindered timely and definitive evaluation of drugs in two-arm trials. Innovative trial designs, especially adaptive platform trials may offer significant efficiency gains to this end. We propose a flexible and scalable multi-arm, multi-stage trial platform where opportunities to participate in a clinical trial can become the default for people with amyotrophic lateral sclerosis., Wong et al. reviewed historical approaches to clinical trials in amyotrophic lateral sclerosis. Lack of sensitive outcome measures, limitations in resources and barriers to trial participation were challenges for timely and definitive evaluation of drugs in two-arm trials. They concluded that future trials should be more flexible, scalable and efficient., Graphical Abstract Graphical Abstract

In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common—along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE’s] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015–2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews., A cross sectional study of a sample of recent preclinical systematic reviews reveals deficiencies in reporting and provides the justification and evidence to inform the development of specific guidelines for conducting and reporting preclinical systematic reviews.

Metaresearch is a scientific field involving the study of research itself. It has been applied to clinical trials since the 1980s,1 but has only become an emerging discipline over the last decade in the preclinical field. The primary tool of metaresearch is the systematic review, which uses predefined methods to provide a transparent and comprehensive summary of the evidence relating to a research question. A systematic review is defined as ‘a review that uses explicit, systematic methods to collate and synthesize findings of studies that address a clearly formulated question’.2 Systematic reviews allow for evaluation of methods and comprehensiveness of reporting, to assay likelihood of reproducibility and potential for translatability to subsequent domains of research and can investigate the impact of incentives on primary research. This, in turn, allows for a more rigorous understanding of what makes research reliable, and how research can be improved,3 while driving evidence-based decisions for future research.4 5 Systematic reviews typically comprise several steps. Before beginning a systematic review, it is recommended that the author team develop a protocol, which defines the research question and the methods that will be used to conduct, analyse and report the findings of the review. The research question determines the resources required to complete the review, the broader the question and the larger the field the more resource intensive a review will be. The search strategy is developed to identify as much potentially relevant literature as possible, often involving searching of multiple databases. Following database searches, deduplication (if searching multiple database with overlapping coverage), the unique search results are screened for inclusion or exclusion. Full-text retrieval may be conducted before or after title and abstract citation screening. Metadata including information regarding reported study quality and design, and outcome data are then extracted from the included …

Introduction and objective The Western diet that comprises high levels of long-chain saturated fats and sugar is associated not only with metabolic disorders such as obesity and type 2 diabetes but also has been recently linked to brain changes and cognitive dysfunction. However, in animal studies, reported effects are variable, and the mechanisms underlying these effects are unclear. In the proposed review, we aim to summarise the diverse evidence of the effects of so-called ‘high-fat’ and ketogenic diets on behavioural measures of cognition in postweaning mice and rats, relative to animals on standard diets and to determine potential underlying mechanisms of high-fat diet-induced effects. Search strategy A comprehensive search strategy was designed to retrieve studies reporting use of a high-fat or ketogenic diet in postweaning mice and rats that included cognitive assessments. Three databases (Medline, SCOPUS and Web of Science) were searched and 4487 unique references were retrieved. Screening and annotation Studies were screened for inclusion by two independent reviewers, with 330 studies retained for analysis. Characteristics of disease model choice, experimental design, intervention use and outcome assessment are to be extracted using the Systematic Review Facility (http://syrf.org.uk/) tool. Studies will be assessed for study quality and risk of bias and confidence of mechanistic involvement. Data management and reporting For cognitive outcomes, effect sizes will be calculated using normalised mean difference and summarised using a random effects model. The contribution of potential sources of heterogeneity to the observed effects of diet on cognition will be assessed using multivariable meta-regression, with partitioning of heterogeneity as a sensitivity analysis. A preliminary version of this protocol was published on 9 April 2019 on the Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies website (http://www.dcn.ed.ac.uk/camarades/research.html%23protocols). Ethics and dissemination No ethical approval is required as there are no subjects in the proposed study.