Search

International audience; Data sharing is gradually becoming a requirement across all fields of science, owing to its key benefits in verifying the reproducibility of findings and reusing existent data for new purposes. Although meta-research studies are complex, time-consuming and hinge on the availability of data produced and curated by others, there has been little focus on how they make their own data available. This is in stark contrast with the heightened attention data sharing has received in clinical research. Yet, as secondary data users par excellence, meta-researchers are ethically bound to both improving and evaluating data sharing practices, as well as correctly sharing their own data. We contrast particularities of data sharing in meta-research and clinical research, such as benefits, barriers, inadequate and potentially pervasive sharing practices. We conclude with an array of concrete and tailored recommendations for improvement.

ObjectiveRegistered reports relate to a new publication of a peer-review of the protocol before the start of the study, followed by an in-principle acceptance by the journal before the study starts. We aimed to describe randomized controlled trials (RCTs) in the clinical field published as registered reports.Study design and settingThis cross-sectional study (registration:https://osf.io/zf53p/) included registered report results for RCTs, identified on PubMed/Medline and on a list compiled by the Center for Open Science. It explored the proportion of reports that received in-principle acceptance (and/or published a protocol before inclusion of the first patient) and changes in the primary outcome.ResultsA total of 93 RCT publications identified as registered reports were included. All but one were published in the same journal group. The date of the in-principle acceptance was never documented. For most of these reports (79/93, 84.9 %) a protocol was published after the date of inclusion of the first patient. A change in the primary outcome was noted in 40/93 (44%) of these publications. Three out of the 40 (33%) mentioned this change.ConclusionsRandomized controlled trials in the clinical field identified as registered reports were rare, they originated from a single journal group and did not comply with the basic features of this format.Protocol registrationhttps://osf.io/zf53p/What is new ?The registered report format for clinical randomized controlled (RCTs) trials is still marginal and few journals make use of it.The clinical RCTs identified as registered reports were from a single journal group and did not necessarily comply with the basic features of this format, and common biases may thus persist.To improve research trustworthiness, more efforts need to be made by Journal publishers, trial funders, etc. for the implementation of this format for clinical RCTs.

International audience

International audience; AIM: The placebo effect and the specific effect are often thought to add up (additive model). Whether additivity holds can dramatically influence the external validity of a trial. This assumption of additivity was tested by Kleijnen et al. in 1994 but the data produced since then has not been synthetized. In this review, we aimed to systematically review the literature to determine whether additivity held. METHODS: We searched Medline and PsychInfo up to 10/01/2019. Studies using the balanced placebo design (BPD), testing two different strengths of placebos, were included. The presence of interaction was evaluated by comparing each group in BPD with analysis of variance or covariance. RESULTS: 30 studies were included and the overall risk of bias was high: four found evidence of additivity and 16 studies found evidence of interaction (seven had evidence of positive additivity). CONCLUSION: Evidence of additivity between placebo and specific features of treatments was rare in included studies. We suggest interventions for placebo-sensitive ailments should be tested in trials designed to take interactions seriously once an exploratory RCTs has proven their efficacy with sufficient internal validity.

International audience; Data sharing enhances the value of medical research and builds trust in clinical trials, but more biomedical researchers need to be trained in these approaches, which include meta-research, data science and ethical, legal and social issues.

Reproducibility is a fundamental premise of trustworthy science. Since 2005 and the Ioannidis JP(1) seminal paper suggesting that most published research findings may be false, there is a growing narrative of a reproducibility crisis in Science (2). Reproducibility is a complex concept that encompasses 3 critical aspects (3): 1/ Method reproducibility (“the provision of enough detail about study procedures and data so the same procedures could, in theory or in actuality, be exactly repeated”), 2/ results reproducibility (“obtaining the same results from the conduct of an independent study whose procedures are as closely matched to the original experiment as possible”) and 3/ inference reproducibility (“drawing of qualitatively similar conclusions from either an independent replication of a study or a reanalysis of the original study”). Sloppy methods, failure to control for bias, flexibility in statistical analysis, poor reporting and publications are some common threats to research reproducibility (4). Many initiatives are expected to improve the reproducibility of scientific research. For instance, the EQUATOR Network (5) develops reporting guidelines such as CONSORT (6) for Randomised Controlled Trials (RCTs) and PRISMA(7) for systematic review and meta-analyses. By improving the reporting of the methods of a given study (methods reproducibility), those guidelines are expected to facilitate results reproducibility by allowing independent researchers to easily implement a replication study. There is some evidence about the positive impact of those guidelines on methods reproducibility but their impact on results reproducibility has to be proven (8). It is quite challenging to measure the concept of results reproducibility. Large reproducibility projects (9) remain rare, very difficult to run, time consuming. Despite being of major importance, this is a research domain that is still limited to a few zealous statisticians. One can also ask who really cares about reproducibility? Within the current research ecosystem, new -innovative- results are incentivized much more than the careful, boring -and generally failed- attempts to reproduce those results. However, there is one thing that everyone has an interest in reproducing, especially at Christmas: it is the cute and greedy character of the small Christmas tree meringues. The primary objective of the study is to prove that a reporting guideline improves the reproducibility of cooking Christmas tree meringues.

International audience; [No abstract available]

International audience

International audience; Evidence-based medicine is the cornerstone of shared-decision making in healthcare today. The public deserves clear, transparent and trust-worthy information on drug efficacy. Yet today, many drugs are prescribed and used without solid evidence of efficacy. Clinical trials and randomized clinical trials (RCTs) are the best method to evaluate drug efficacy and side effects. In a shared medical decision-making approach, general practitioners need drug assessment to be based on patient-important outcomes. The aim of project Rebuild the Evidence Base (REB) is to bridge the gap between the data needed in clinical practice and the data available from clinical research. The drugs will be assessed on clinical patient important outcomes and for a population. Using the Cochrane tools, we propose to analyse for each population and outcome : 1) a meta-analysis based on RCTs with a low risk of bias overall ; 2) an evaluate of results of confirmatory RCTs; 3) a statistical analysis of heterrogeneity between RCTs, and 4) an analysis of publication bias. Depending on the results of these analyses, the evidence will be categorised in 4 different levels: firm evidence, evidence (to be confirmed), signal or absence of evidence. Project REB proposes a method for reading and interpreting randomized clinical trials and their meta-analysis to produce quality data for general practitioners to focus on benefit-risk assessment in the interest of patients. If this data does not exist, it could enable clinical research to better its aim.

We will examine the extent of variation between modelling studies and with real-world data about the proportion of the population compliant with lockdowns (i.e., that drastically reduces contact as expected) and whether this variation is associated with "vibration of effects" in terms of number of deaths and active cases in simulations based on a recent model examining the effects of various social distancing interventions.

Meta-research project

The COVID-19 outbreak has prompted an unprecedented use of preprint servers requiring to assess the integrity, handling and impact of this very fast communication of information between health science and public.

As part of a global research program on Reproducibility in Therapeutic Research (ReiTheR, funded by the French National Research Agency), we designed this cross-sectional study to assess the inferential reproducibility after reanalysis of a sample of RCTs registered on selected data-sharing platforms.

A protocol for a cross sectional analysis of published re-use from 3 major repositories

Protocol

Different methodological choices such as inclusion/exclusion criteria and analytical models can yield different results and inferences when meta-analyses are performed. For example, it has been shown in network meta-analyses that even the consideration of which nodes are eligible and/or whether a placebo should be considered can yield very different results. Vibration of effects (VoE) describes the extent to which an effect may change under multiple distinct analyses, such as different model specifications in epidemiological research or in meta-analyses. We propose applying the VoE concept for examining the range of treatment effects that can be obtained in mixed treatment comparisons of antidepressants in major depressive disorders when considering all possible combinations of overlapping networks. We will describe the range of different results arising from different network geometries and propose a visual tool to assess the range of uncertainties.

Survey of studies in journal with data-sharing policies

All extracted data journal and article-level data for the manuscript Nutu, D., Gentili, C., Naudet, F., Cristea, I.A. (2019) "Open Science Practices in Clinical Psychology Journals: An Audit Study". Journal of Abnormal Psychology.

Correlation between gifts from pharmaceutical companies to French medical academics and their scientific productivity patterns: a protocol for a retrospective study using the French Transparency in Healthcare Alexandre Scanff1, Ioana Cristea2,3, Pierre Etienne Savourat1, Thomas Clément1, Benjamin Bastian1, Marion Taburet1, Alain Braillon4, Florian Naudet, Professor1,* 1: Univ Rennes, CHU Rennes, Inserm, CIC 1414 (Centre d’Investigation Clinique de Rennes), France 2: Department of Clinical Psychology and Psychotherapy, Babes-Bolyai University, St.Republicii 37, 400015 Cluj-Napoca, Romania 3: University of Pavia, Department of Brain and Behavioral Sciences, 27100 Pavia, IT 4: Alcohol Treatment Unit, University Hospital, 80054 Amiens CEDEX 1. France * : corresponding author ; orcid.org/0000-0003-3760-3801 ; mail floriannaudet@gmail.com ; twitter handle /naudetflorian The scientific impact of academics is appraised with their ability to produce knowledge. Funding, promotion of individuals and survival of their team depends on their publication records and some quantitative citations metrics. Many projects such in genomics [1] can only be conducted in collaboration and new ideas often arise by conversations with colleagues. Collaboration driven by industrialists may also have an impact on researchers' productivity at the price of potential financials conflicts of interest (COIs). Prolific academics are often identified and pushed by industrialists as "Key Opinion Leaders" on the ground of their publication records, lectures, and their participation on different groups [2]. For example, in the field of oncology, financial COIs were correlated with metrics of academic reputation and success [3]. And indeed, committees hiring academics use indicators such as scientific productivity (i.e. number of published papers) and quality. For quality, two proxies are generally used: a) the journal impact factor (JIF) and b) the Hirsch Index (h-index), corresponding to the largest number h of papers one has published that have been cited at least h times. [4] We aim to investigate the relationship presently used measures of academic success (publications/citations/h-index) with the potential financial COIs. Methods Study design and participants The project will be based on a protocol that will be registered on the Open Science Framework prior any data collection. In France, academic duties require both an active clinical activity and a research activity. While clinical skills are only evaluated qualitatively, assessment of research activity entails publication metrics, which are easy to quantify and investigate. This study will be conducted and reported to the Standards for Reporting Diagnostic Accuracy. [5] One researcher will identify all French Medical Professors and associated professors (and their academic discipline) from lists of the different universities and the “annuaire santé” that also details sex and medical specialty (i.e. academic discipline). Then we will extract from online databases (e.g. Google Scholar, Web of Science, Scopus) the year of their first paper (an indicator of seniority) and quantitative indicators of productivity for the 2014-2019 period (h-index over the period, number of papers and number of citations). For each academic, the amount of money declared by the industry during the 2014-2019 period will be extracted from the French database registering all direct gifts (e.g., meals or congress-related hotel, and travel costs) perceived by healthcare players. This open access database was created in 2012 (https://www.transparence.sante.gouv.fr/) and the industry is required by law to report. We will use only the “advantage” table, comprising all industry funding which in theory is not associated with a contractual obligation of the recipient. Two more tables exist, which report “conventions” and “remunerations” when there is a contractual obligation for the recipient, but reporting the monetary value was not mandatory and poorly implemented during the all the study period, and thus the amount is almost-always lacking. An automatic search will be developed, possibly completed by manual checks to explore the possibility of misidentification. Ethics We are seeking the opinion of an ethics committee, namely that of the CNGE Ethics Committee. The procedure toward protection of persons has been validated by the Data Privacy Officer of Rennes 1 University. In view of the large number of subjects studied, we cannot afford to contact all of them. Moreover, this step would risk adding a social desirability bias to the results of our study that may invalidate the study results. Data collected are not considered as sensitive data by the French law. In addition, general information will be made publicly available on the Reither website (https://www.reither.org/) detailing the protocol, the use that will be made of the data, with a contact information (Pr Florian Naudet: floriannaudet@gmail.com). This information will also be detailed in the thesis references once it has been published to indicate to the interested parties the right of access to any data concerning them. Indexes The f-index will be defined as the sum of gifts received by each academic in the 2014-2019 period (meals, and/or travel/accommodation expenses). The gold standard used for the purpose of the study will be high scientific productivity defined for an academic as an h-index over the period 2014-2019 in the top quartile of h-indexes in the group analysed and in subgroups defined by all academic discipline for subgroup analyses. As no gold standard exists but institutions need to take decisions under constraints of limited time and budgets, we opted for this standard. It reflects the fact that in the current system effectively only a few top researchers can expect promotion. Outcomes and analysis First, a descriptive analysis will be performed, consisting of numbers and percentages for categoric outcomes, and means, standard-deviations for quantitative outcomes. Then, Pearson’s correlations will be used to explore the association between indicators and several characteristics of the academics, after logarithmic transformation (index value+1), if needed. Receiver Operating Characteristic (ROC) curves will be drawn by plotting sensitivity vs. one minus specificity and the optimal threshold will be determined according to Youden’s J measure. The main outcome for this study will be the diagnostic properties (sensitivity, specificity, positive predictive value, negative predictive value) of the f-index to objectively reflect high scientific productivity. [6] Finally, to account for possible variations according to subspecialty, subgroup analyses (f-index, h-index, correlation between f-index and h-index, optimal threshold) will be explored by academic discipline separately. 95% confidence intervals for test parameters will be estimated by bootstrap procedure. An exploratory analysis of the effect of sex and year of first paper on the f-index will be realized by linear mixed model, using a random intercept for academic speciality. REFERENCES: 1. Lehner T, Senthil G, Addington AM. Convergence of advances in genomics, team science, and repositories as drivers of progress in psychiatric genomics. Biological psychiatry 2015;77(1):6-14. 2. Moynihan R. Key opinion leaders: independent experts or drug representatives in disguise? bmj 2008;336(7658):1402-03. 3. Lammers A, Edmiston J, Kaestner V, et al. Financial Conflict of Interest and Academic Influence Among Experts Speaking on Behalf of the Pharmaceutical Industry at the US Food and Drug Ad-ministration's Oncologic Drugs Advisory Committee Meetings. Mayo Clinic proceedings 2017;92(7):1164-66. doi: 10.1016/j.mayocp.2017.04.014 [published Online First: 2017/07/10] 4. Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A 2005;102:16569–72. doi:10.1073/pnas.0507655102 5. Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015;351:h5527. doi:10.1136/bmj.h5527 6. Perkins NJ, Schisterman EF. The Inconsistency of “Optimal” Cut-points Using Two ROC Based Criteria. Am J Epidemiol 2006;163:670–5. doi:10.1093/aje/kwj063

The aim of this study is to compare the actual efficiency of the different dose-escalation designs by focusing on the therapeutic class of small-molecule tyrosine kinase inhibitors.