Search

Abstract Background Researchers performing high-quality systematic reviews search across multiple databases to identify relevant evidence. However, the same publication is often retrieved from several databases. Identifying and removing such duplicates (“deduplication”) can be extremely time-consuming, but failure to remove these citations can lead to the wrongful inclusion of duplicate data. Many existing tools are not sensitive enough, lack interoperability with other tools, are not freely accessible, or are difficult to use without programming knowledge. Here, we report the performance of our Automated Systematic Search Deduplicator (ASySD), a novel tool to perform automated deduplication of systematic searches for biomedical reviews. Methods We evaluated ASySD’s performance on 5 unseen biomedical systematic search datasets of various sizes (1845–79,880 citations). We compared the performance of ASySD with EndNote’s automated deduplication option and with the Systematic Review Assistant Deduplication Module (SRA-DM). Results ASySD identified more duplicates than either SRA-DM or EndNote, with a sensitivity in different datasets of 0.95 to 0.99. The false-positive rate was comparable to human performance, with a specificity of > 0.99. The tool took less than 1 h to identify and remove duplicates within each dataset. Conclusions For duplicate removal in biomedical systematic reviews, ASySD is a highly sensitive, reliable, and time-saving tool. It is open source and freely available online as both an R package and a user-friendly web application.

Abstract Objective Software developed using artificial intelligence may automatically identify arterial occlusion and provide collateral vessel scoring on CT angiography (CTA) performed acutely for ischemic stroke. We aimed to assess the diagnostic accuracy of e‐CTA by Brainomix™ Ltd by large‐scale independent testing using expert reading as the reference standard. Methods We identified a large clinically representative sample of baseline CTA from 6 studies that recruited patients with acute stroke symptoms involving any arterial territory. We compared e‐CTA results with masked expert interpretation of the same scans for the presence and location of laterality‐matched arterial occlusion and/or abnormal collateral score combined into a single measure of arterial abnormality. We tested the diagnostic accuracy of e‐CTA for identifying any arterial abnormality (and in a sensitivity analysis compliant with the manufacturer's guidance that software only be used to assess the anterior circulation). Results We include CTA from 668 patients (50% female; median: age 71 years, NIHSS 9, 2.3 h from stroke onset). Experts identified arterial occlusion in 365 patients (55%); most (343, 94%) involved the anterior circulation. Software successfully processed 545/668 (82%) CTAs. The sensitivity, specificity and diagnostic accuracy of e‐CTA for detecting arterial abnormality were each 72% (95% CI = 66–77%). Diagnostic accuracy was non‐significantly improved in a sensitivity analysis excluding occlusions from outside the anterior circulation (76%, 95% CI = 72–80%). Interpretation Compared to experts, the diagnostic accuracy of e‐CTA for identifying acute arterial abnormality was 72–76%. Users of e‐CTA should be competent in CTA interpretation to ensure all potential thrombectomy candidates are identified.

Background: Anhedonia is a key symptom of depression, and it has been suggested as a potential target for future individualised treatments. However, much is unknown about how interventions enhancing dopaminergic pathways may affect anhedonia symptoms in the context of depression. Methods: We will perform independent searches in multiple electronic databases to identify clinical and animal experimental studies on pro-dopaminergic interventions in individuals with depression or animal models for depression. The primary outcomes will be overall anhedonia symptoms and their behavioural proxies in animals. Secondary outcomes will include side effects and neurobiological measures. At least two independent reviewers will conduct the study selection, data extraction, and risk of bias assessments using pre-defined tools according to each record’s study design. We will develop ontologies to facilitate study identification and data extraction. We will synthesise data from clinical and animal studies separately. If appropriate, we will use random-effects meta-analyses, or synthesis without meta-analyses. We will investigate study characteristics as potential sources of heterogeneity. We will evaluate the confidence in the evidence for each outcome and source of evidence, considering the summary of the association, potential concerns regarding internal and external validity, and reporting biases. When multiple sources of evidence are available for an outcome, we will draw an overall conclusion in a triangulation meeting involving a multidisciplinary team of experts. We plan updates of the review every 6 months, and any future modifications to the protocol will be documented. We will co-produce this review with multiple stakeholders. PROSPERO registration: CRD42023451821

In anxiety, depression and psychosis, there has been frustratingly slow progress in developing novel therapies that make a substantial difference in practice, as well as in predicting which treatments will work for whom and in what contexts. To intervene early in the process and deliver optimal care to patients, we need to understand the underlying mechanisms of mental health conditions, develop safe and effective interventions that target these mechanisms, and improve our capabilities in timely diagnosis and reliable prediction of symptom trajectories. Better synthesis of existing evidence is one way to reduce waste and improve efficiency in research towards these ends. Living systematic reviews produce rigorous, up-to-date and informative evidence summaries that are particularly important where research is emerging rapidly, current evidence is uncertain and new findings might change policy or practice. Global Alliance for Living Evidence on aNxiety, depressiOn and pSychosis (GALENOS) aims to tackle the challenges of mental health science research by cataloguing and evaluating the full spectrum of relevant scientific research including both human and preclinical studies. GALENOS will also allow the mental health community—including patients, carers, clinicians, researchers and funders—to better identify the research questions that most urgently need to be answered. By creating open-access datasets and outputs in a state-of-the-art online resource, GALENOS will help identify promising signals early in the research process. This will accelerate translation from discovery science into effective new interventions for anxiety, depression and psychosis, ready to be translated in clinical practice across the world.

Abstract Background Natural language processing could assist multiple tasks in systematic reviews to reduce workflow, including the extraction of PICO elements such as study populations, interventions, comparators and outcomes. The PICO framework provides a basis for the retrieval and selection for inclusion of evidence relevant to a specific systematic review question, and automatic approaches to PICO extraction have been developed particularly for reviews of clinical trial findings. Considering the difference between preclinical animal studies and clinical trials, developing separate approaches is necessary. Facilitating preclinical systematic reviews will inform the translation from preclinical to clinical research. Methods We randomly selected 400 abstracts from the PubMed Central Open Access database which described in vivo animal research and manually annotated these with PICO phrases for Species, Strain, methods of Induction of disease model, Intervention, Comparator and Outcome. We developed a two-stage workflow for preclinical PICO extraction. Firstly we fine-tuned BERT with different pre-trained modules for PICO sentence classification. Then, after removing the text irrelevant to PICO features, we explored LSTM-, CRF- and BERT-based models for PICO entity recognition. We also explored a self-training approach because of the small training corpus. Results For PICO sentence classification, BERT models using all pre-trained modules achieved an F1 score of over 80%, and models pre-trained on PubMed abstracts achieved the highest F1 of 85%. For PICO entity recognition, fine-tuning BERT pre-trained on PubMed abstracts achieved an overall F1 of 71% and satisfactory F1 for Species (98%), Strain (70%), Intervention (70%) and Outcome (67%). The score of Induction and Comparator is less satisfactory, but F1 of Comparator can be improved to 50% by applying self-training. Conclusions Our study indicates that of the approaches tested, BERT pre-trained on PubMed abstracts is the best for both PICO sentence classification and PICO entity recognition in the preclinical abstracts. Self-training yields better performance for identifying comparators and strains.

Objectives Motor neuron disease (MND) is an incurable progressive neurodegenerative disease with limited treatment options. There is a pressing need for innovation in identifying therapies to take to clinical trial. Here, we detail a systematic and structured evidence-based approach to inform consensus decision making to select the first two drugs for evaluation in Motor Neuron Disease-Systematic Multi-arm Adaptive Randomised Trial (MND-SMART: NCT04302870), an adaptive platform trial. We aim to identify and prioritise candidate drugs which have the best available evidence for efficacy, acceptable safety profiles and are feasible for evaluation within the trial protocol.Methods We conducted a two-stage systematic review to identify potential neuroprotective interventions. First, we reviewed clinical studies in MND, Alzheimer’s disease, Huntington’s disease, Parkinson’s disease and multiple sclerosis, identifying drugs described in at least one MND publication or publications in two or more other diseases. We scored and ranked drugs using a metric evaluating safety, efficacy, study size and study quality. In stage two, we reviewed efficacy of drugs in MND animal models, multicellular eukaryotic models and human induced pluripotent stem cell (iPSC) studies. An expert panel reviewed candidate drugs over two shortlisting rounds and a final selection round, considering the systematic review findings, late breaking evidence, mechanistic plausibility, safety, tolerability and feasibility of evaluation in MND-SMART.Results From the clinical review, we identified 595 interventions. 66 drugs met our drug/disease logic. Of these, 22 drugs with supportive clinical and preclinical evidence were shortlisted at round 1. Seven drugs proceeded to round 2. The panel reached a consensus to evaluate memantine and trazodone as the first two arms of MND-SMART.Discussion For future drug selection, we will incorporate automation tools, text-mining and machine learning techniques to the systematic reviews and consider data generated from other domains, including high-throughput phenotypic screening of human iPSCs.

Abstract Increasing awareness of problems with the reproducibility and integrity of research led the UK Parliament Science and Technology Committee to launch, in July 2021, an inquiry into reproducibility and research integrity. We recognise at least four potential reasons why attempts to replicate a research finding may be unsuccessful: false positive statistical analyses, low generalisability of findings, suboptimal study designs (research integrity), and deliberate malfeasance (researcher integrity). It is important to make a distinction between the contributions of research integrity and of researcher integrity to the reproducibility crisis. While the impact of an individual instance of compromised researcher integrity is substantial, the aggregate impact of more prevalent problems with research integrity is likely much greater. The research community will be most efficient when failed replication efforts are never due to issues of research integrity or of researcher integrity, as this would allow focus on the scientific reasons for why two apparently similar experiments should reach different conclusions. We discuss the role of funders, institutions and government in addressing the “reproducibility crisis” before considering which interventions might have a positive impact on academia’s approach to reproducible research, and a possible role for a committee on research integrity.

Abstract The adoption and incentivisation of open and transparent research practices is critical in addressing issues around research reproducibility and research integrity. These practices will require training and funding. Individuals need to be incentivised to adopt open and transparent research practices (e.g., added as desirable criteria in hiring, probation, and promotion decisions, recognition that funded research should be conducted openly and transparently, the importance of publishers mandating the publication of research workflows and appropriately curated data associated with each research output). Similarly, institutions need to be incentivised to encourage the adoption of open and transparent practices by researchers. Research quality should be prioritised over research quantity. As research transparency will look different for different disciplines, there can be no one-size-fits-all approach. An outward looking and joined up UK research strategy is needed that places openness and transparency at the heart of research activity. This should involve key stakeholders (institutions, research organisations, funders, publishers, and Government) and crucially should be focused on action. Failure to do this will have negative consequences not just for UK research, but also for our ability to innovate and subsequently commercialise UK-led discovery.

Abstract There are many factors that contribute to the reproducibility and replicability of scientific research. There is a need to understand the research ecosystem, and improvements will require combined efforts across all parts of this ecosystem. National structures can play an important role in coordinating these efforts, working collaboratively with researchers, institutions, funders, publishers, learned societies and other sectoral organisations, and providing a monitoring and reporting function. Whilst many new ways of working and emerging innovations hold a great deal of promise, it will be important to invest in meta-research activity to ensure that these approaches are evidence based, work as intended, and do not have unintended consequences. Addressing reproducibility will require working collaboratively across the research ecosystem to share best practice and to make the most effective use of resources. The UK Reproducibility Network (UKRN) brings together Local Networks of researchers, Institutions, and External Stakeholders (funders, publishers, learned societies and other sectoral organisations), to coordinate action on reproducibility and work to ensure the UK retains its place as a centre for world-leading research. This activity is coordinated by the UKRN Steering Group. We consider this structure as valuable, bringing together a range of voices at a range of levels to support the combined efforts required to enact change.

Abstract Background In pre-clinical research, systematic reviews have the potential to mitigate translational challenges by facilitating understanding of how pre-clinical studies can inform future clinical research. Yet their conduct is encumbered by heterogeneity in the outcomes measured and reported, and those outcomes may not always relate to the most clinically important outcomes. We aimed to systematically review outcomes measured and reported in pre-clinical in vivo studies of pharmacological interventions to treat high blood glucose in mouse models of type 2 diabetes. Methods A systematic review of pre-clinical in vivo studies of pharmacological interventions aimed at addressing elevated blood glucose in mouse models of type 2 diabetes was completed. Studies were screened for eligibility and outcomes extracted from the included studies. The outcomes were recorded verbatim and classified into outcome domains using an existing outcome taxonomy. Outcomes were also compared to those identified in a systematic review of registered phase 3/4 clinical trials for glucose lowering interventions in people with type 2 diabetes. Results Review of 280 included studies identified 532 unique outcomes across 19 domains. No single outcome, or domain, was measured in all studies and only 132 (21%) had also been measured in registered phase 3/4 clinical trials. A core outcome set, representing the minimum that should be measured and reported, developed for type 2 diabetes effectiveness clinical trials includes 18 core outcomes, of these 12 (71%) outcomes were measured and reported in one or more of the included pre-clinical studies. Conclusions There is heterogeneity of outcomes reported in pre-clinical research. Harmonisation of outcomes across the research pathway using a core outcome set may facilitate interpretation, evidence synthesis and translational success, and may contribute to the refinement of the use of animals in research. Systematic review registration: The study was prospectively registered on the PROSPERO Database, registration number CRD42018106831

Abstract Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the “ARRIVE Essential 10,” which constitutes the minimum requirement, and the “Recommended Set,” which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

Abstract Background Targeted temperature management (TTM) of 32 to 34 °C has been the standard treatment for out-of-hospital cardiac arrest since clinical trials in 2002 indicated benefit on survival and neurological outcome. In 2013, a clinical trial showed no difference in outcome between TTM of 33 °C and TTM of 36 °C. In this meta-analysis, we investigate the evidence for TTM in animal models of cardiac arrest. Methods We searched PubMed and EMBASE for adult animal studies using TTM as a treatment in different models of cardiac arrest or global brain ischemia which reported neurobehavioural outcome, brain histology or mortality. We used a random effects model to calculate estimates of efficacy and assessed risk of bias using an adapted eight-item version of the Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies (CAMARADES) quality checklist. We also used a scoring system based on the recommendations of the Stroke Treatment Academic Industry Roundtable (STAIR), to assess the scope of testing in the field. Included studies which investigated a post-ischemic induction of TTM had their treatment regimens characterized with regard to depth, duration and time to treatment and scored against the modified STAIR criteria. Results The initial and updated search generated 17809 studies after duplicate removal. One hundred eighty-one studies met the inclusion criteria, including data from 1,787, 6,495 and 2,945 animals for neurobehavioural, histological and mortality outcomes, respectively. TTM was favoured compared to control for all outcomes. TTM was beneficial using short and prolonged cooling, deep and moderate temperature reduction, and early and delayed time to treatment. Median [IQR] study quality was 4 [3 to 6]. Eighteen studies checked seven or more of the eight CAMARADES quality items. There was no clear correlation between study quality and efficacy for any outcome. STAIR analysis identified 102 studies investigating post-ischemic induction of TTM, comprising 147 different treatment regimens of TTM. Only 2 and 8 out of 147 regimens investigated comorbid and gyrencephalic animals, respectively. Conclusions TTM is beneficial under most experimental conditions in animal models of cardiac arrest or global brain ischemia. However, research on gyrencephalic species and especially comorbid animals is uncommon and a possible translational gap. Also, low study quality suggests risk of bias within studies. Future animal research should focus on mimicking the clinical scenario and employ similar rigour in trial design to that of modern clinical trials.

In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common-along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE's] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015-2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.

Abstract Research synthesis is the process of bringing together findings and attributes from different publications, for example, to give a more complete description of phenomena than is usually possible in a single work. We bring the Research Synthesis Series to BMC Biology to promote meta-analyses, other research syntheses including meta-research studies, and research synthesis methodologies in biology, facilitating their dissemination to broader communities.

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited

Background: The process of translating preclinical findings into a clinical setting takes decades. Previous studies have suggested that only 5-10% of the most promising preclinical studies are successfully translated into viable clinical applications. The underlying determinants of this low success rate (e.g. poor experimental design, suboptimal animal models, poor reporting) have not been examined in an empirical manner. Our study aims to determine the contemporary success rate of preclinical-to-clinical translation, and subsequently determine if an association between preclinical study design and translational success/failure exists. Methods: Established systematic review methodology will be used with regards to the literature search, article screening and study selection process. Preclinical, basic science studies published in high impact basic science journals between 1995 and 2015 will be included. Included studies will focus on publicly available interventions with potential clinical promise. The primary outcome will be successful clinical translation of promising therapies - defined as the conduct of at least one Phase II trial (or greater) with a positive finding. A case-control study will then be performed to evaluate the association between elements of preclinical study design and reporting and the likelihood of successful translation. Discussion: This study will provide a comprehensive analysis of the therapeutic translation from the laboratory bench to the bedside. Importantly, any association between factors of study design and the success of translation will be identified. These findings may inform future research teams attempting preclinical-to-clinical translation. Results will be disseminated to identified knowledge users that fund/support preclinical research.

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited

Improving the reproducibility of biomedical research is a major challenge. Transparent and accurate reporting is vital to this process; it allows readers to assess the reliability of the findings and repeat or build upon the work of other researchers. The ARRIVE guidelines (Animal Research: Reporting In Vivo Experiments) were developed in 2010 to help authors and journals identify the minimum information necessary to report in publications describing in vivo experiments. Despite widespread endorsement by the scientific community, the impact of ARRIVE on the transparency of reporting in animal research publications has been limited. We have revised the ARRIVE guidelines to update them and facilitate their use in practice. The revised guidelines are published alongside this paper. This explanation and elaboration document was developed as part of the revision. It provides further information about each of the 21 items in ARRIVE 2.0, including the rationale and supporting evidence for their inclusion in the guidelines, elaboration of details to report, and examples of good reporting from the published literature. This document also covers advice and best practice in the design and conduct of animal studies to support researchers in improving standards from the start of the experimental design process through to publication.

Background: Artificial intelligence-based software may automatically detect ischaemic stroke lesions and provide an Alberta Stroke Program Early CT score (ASPECTS) on CT, and identify arterial occlusion and provide a collateral score on CTA. Large-scale independent testing will inform clinical use, but is lacking. We aim to test e-ASPECTS and e-CTA (Brainomix, Oxford UK) using CT scans obtained from a range of clinical studies. Methods: Using prospectively collected baseline CT and CTA scans from 10 national/international clinical stroke trials or registries (total >6600 patients), we will select a large clinically representative sample for testing e-ASPECTS and e-CTA compared to previously acquired independent expert human interpretation (reference standard). Our primary aims are to test agreement between software-derived and masked human expert ASPECTS, and the diagnostic accuracy of e-ASPECTS for identifying all causes of stroke symptoms using follow-up imaging and final clinical opinion as diagnostic ground truth. Our secondary aims are to test when and why e-ASPECTS is more or less accurate, or succeeds/fails to produce results, agreement between e-CTA and human expert CTA interpretation, and repeatability of e-ASPECTS/e-CTA results. All testing will be conducted on an intention-to-analyse basis. We will assess agreement between software and expert-human ratings and test the diagnostic accuracy of software. Conclusions: RITeS will provide comprehensive, robust and representative testing of e-ASPECTS and e-CTA against the current gold-standard, expert-human interpretation.

We report a systematic review and meta-analysis of research using animal models of chemotherapy-induced peripheral neuropathy (CIPN). We systematically searched 5 online databases in September 2012 and updated the search in November 2015 using machine learning and text mining to reduce the screening for inclusion workload and improve accuracy. For each comparison, we calculated a standardised mean difference (SMD) effect size, and then combined effects in a random-effects meta-analysis. We assessed the impact of study design factors and reporting of measures to reduce risks of bias. We present power analyses for the most frequently reported behavioural tests; 337 publications were included. Most studies (84%) used male animals only. The most frequently reported outcome measure was evoked limb withdrawal in response to mechanical monofilaments. There was modest reporting of measures to reduce risks of bias. The number of animals required to obtain 80% power with a significance level of 0.05 varied substantially across behavioural tests. In this comprehensive summary of the use of animal models of CIPN, we have identified areas in which the value of preclinical CIPN studies might be increased. Using both sexes of animals in the modelling of CIPN, ensuring that outcome measures align with those most relevant in the clinic, and the animal's pain contextualised ethology will likely improve external validity. Measures to reduce risk of bias should be employed to increase the internal validity of studies. Different outcome measures have different statistical power, and this can refine our approaches in the modelling of CIPN.

Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we integrated and customised two existing JavaScript libraries to create a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison with participants’ standard practice for extracting data from graphs in PDF documents. Results: We found that the customised graphical data extraction tool is not inferior to users’ (N=10) prior standard practice. Our study was not designed to show superiority, but suggests that, on average, participants saved around 6 minutes per graph using the new tool, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.

Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we integrated and customised two existing JavaScript libraries to create a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison to standard practice. Results: We found that the customised graphical data extraction tool is not inferior to users’ prior preferred current approaches. Our study was not designed to show superiority, but suggests that there may be a saving in time of around 6 minutes per graph, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.

Addressing the common problems that researchers encounter when designing and analysing animal experiments will improve the reliability of in vivo research. In this article, the Experimental Design Assistant (EDA) is introduced. The EDA is a web-based tool that guides the in vivo researcher through the experimental design and analysis process, providing automated feedback on the proposed design and generating a graphical summary that aids communication with colleagues, funders, regulatory authorities, and the wider scientific community. It will have an important role in addressing causes of irreproducibility.

Evaluation of preclinical evidence prior to initiating early-phase clinical studies has typically been performed by selecting individual studies in a non-systematic process that may introduce bias. Thus, in preparation for a first-in-human trial of mesenchymal stromal cells (MSCs) for septic shock, we applied systematic review methodology to evaluate all published preclinical evidence. We identified 20 controlled comparison experiments (980 animals from 18 publications) of in vivo sepsis models. Meta-analysis demonstrated that MSC treatment of preclinical sepsis significantly reduced mortality over a range of experimental conditions (odds ratio 0.27, 95% confidence interval 0.18–0.40, latest timepoint reported for each study). Risk of bias was unclear as few studies described elements such as randomization and no studies included an appropriately calculated sample size. Moreover, the presence of publication bias resulted in a ~30% overestimate of effect and threats to validity limit the strength of our conclusions. This novel prospective application of systematic review methodology serves as a template to evaluate preclinical evidence prior to initiating first-in-human clinical studies.

Incomplete reporting of study methods and results has become a focal point for failures in the reproducibility and translation of findings from preclinical research. Here we demonstrate that incomplete reporting of preclinical research is not limited to a few elements of research design, but rather is a broader problem that extends to the reporting of the methods and results. We evaluated 47 preclinical research studies from a systematic review of acute lung injury that use mesenchymal stem cells (MSCs) as a treatment. We operationalized the ARRIVE (Animal Research: Reporting of In Vivo Experiments) reporting guidelines for pre-clinical studies into 109 discrete reporting sub-items and extracted 5,123 data elements. Overall, studies reported less than half (47%) of all sub-items (median 51 items; range 37-64). Across all studies, the Methods Section reported less than half (45%) and the Results Section reported less than a third (29%). There was no association between journal impact factor and completeness of reporting, which suggests that incomplete reporting of preclinical research occurs across all journals regardless of their perceived prestige. Incomplete reporting of methods and results will impede attempts to replicate research findings and maximize the value of preclinical studies.

We know that clinical trials sponsored by the pharmaceutical industry are likely to exaggerate benefit and minimise harms. But do these biases extend to their sponsorship of non-human animal research? Using systematic review and meta-analysis Bero and colleagues show that, in the case of statins, things are a little more complicated. While the conclusions of industry-sponsored studies were indeed more enthusiastic than warranted by their data, the data themselves painted a picture more conservative than was seen in non-industry-sponsored studies. This behaviour is consistent with maximising the return on investment, seeking robust data before embarking on a clinical trial, and, once that investment has been made, making every effort to "prove" that the drug is safe and effective if this is at all credible. The findings suggest that there is something different about industry-sponsored non-human animal research, perhaps reflecting higher standards than is the case elsewhere. Perhaps the academic community can learn something from our colleagues in the commercial sector.

Background: The process of translating preclinical findings into a clinical setting takes decades. Previous studies have suggested that only 5-10% of the most promising preclinical studies are successfully translated into viable clinical applications. The underlying determinants of this low success rate (e.g. poor experimental design, suboptimal animal models, poor reporting) have not been examined in an empirical manner. Our study aims to determine the contemporary success rate of preclinical-to-clinical translation, and subsequently determine if an association between preclinical study design and translational success/failure exists. Methods: Established systematic review methodology will be used with regards to the literature search, article screening and study selection process. Preclinical, basic science studies published in high impact basic science journals between 1995 and 2015 will be included. Included studies will focus on publicly available interventions with potential clinical promise. The primary outcome will be successful clinical translation of promising therapies - defined as the conduct of at least one Phase II trial (or greater) with a positive finding. A case-control study will then be performed to evaluate the association between elements of preclinical study design and reporting and the likelihood of successful translation. Discussion: This study will provide a comprehensive analysis of the therapeutic translation from the laboratory bench to the bedside. Importantly, any association between factors of study design and the success of translation will be identified. These findings may inform future research teams attempting preclinical-to-clinical translation. Results will be disseminated to identified knowledge users that fund/support preclinical research.