Search

Abstract High-quality cancer research is crucial to both save lives and improve quality of life. Spurious findings, however, impedes these laudable goals by misleading research efforts and creating research waste that is inherently difficult to counteract. Irreproducible research is intrinsically wasteful, and unsustainable over the long term. In this perspective piece, we elucidate the extent of the current replication crisis and the underlying causes, identifying practices that lend themselves to unsustainable spurious findings, and the factors that underpin these practices. Finally we outline some remedies to the problem of irreproducible research, and how we might move towards more sustainable and trustworthy research in biomedical science.

Background Scientific publications have been growing exponentially, contributing to an oversaturated information environment. Quantifying a research output’s impact and reach cannot be solely measured by traditional metrics like citation counts as these have a lag time and are largely focused on an academic audience. There is increasing recognition to consider ‘alternative metrics’ or altmetrics to measure more immediate and broader impacts of research. Better understanding of altmetrics can help researchers better navigate evolving information environments and changing appetites for different types of research. Objectives Our study aims to: 1) analyse the amount and medium of Altmetric coverage of health research produced by Irish organisations (2017 – 2023), identifying changes over time and 2) investigate differences in the amount of coverage between clinical areas (e.g., nutrition vs. neurology). Methods Using Altmetric institutional access, we will gather data on research outputs published 1 January 2017 through 31 December 2023 from active Irish organisations with Research Organisation Registry (ROR) IDs. Outputs will be deduplicated and stratified by their Australian and New Zealand Standard Research Classification relating to ≥1 field of health research: Biological Sciences, Biomedical and Clinical Sciences, Chemical Sciences, Health Sciences, and Psychology. We will clean data using R and perform descriptive analyses, establishing counts and frequencies of coverage by clinical area and medium (e.g., traditional news, X, etc.); data will be plotted on a yearly and quarterly basis where appropriate. Results and Conclusions Improved understanding of one’s information environment can help researchers better navigate their local landscapes and identify pathways for more effective communication to the public. All R code will be made available open-source, allowing researchers to adapt it to evaluate their local landscapes.

Background Scientific publications have been growing exponentially, contributing to an oversaturated information environment. Quantifying a research output’s impact and reach cannot be solely measured by traditional metrics like citation counts as these have a lag time and are largely focused on an academic audience. There is increasing recognition to consider ‘alternative metrics’ or altmetrics to measure more immediate and broader impacts of research. Better understanding of altmetrics can help researchers better navigate evolving information environments and changing appetites for different types of research. Objectives Our study aims to: 1) analyse the amount and medium of Altmetric coverage of health research produced by Irish organisations (2017 – 2023), identifying changes over time and 2) investigate differences in the amount of coverage between clinical areas (e.g., nutrition vs. neurology). Methods Using Altmetric institutional access, we will gather data on research outputs published 1 January 2017 through 31 December 2023 from active Irish organisations with Research Organisation Registry (ROR) IDs. Outputs will be deduplicated and stratified by their Australian and New Zealand Standard Research Classification relating to ≥1 field of health research: Biological Sciences, Biomedical and Clinical Sciences, Chemical Sciences, Health Sciences, and Psychology. We will clean data using R and perform descriptive analyses, establishing counts and frequencies of coverage by clinical area and medium (e.g., traditional news, X, etc.); data will be plotted on a yearly and quarterly basis where appropriate. Results and Conclusions Improved understanding of one’s information environment can help researchers better navigate their local landscapes and identify pathways for more effective communication to the public. All R code will be made available open-source, allowing researchers to adapt it to evaluate their local landscapes.

Abstract The Y-box binding protein-1 (YBX1) gene codes for a multifunctional oncoprotein that is increasingly being linked to the regulations of many aspects of cancer cell biology. Disparities in treatment outcomes between male and female cancer patients are increasingly reported. This study aimed to examine the relationship between YBX1 expression and overall survival in male and female patients with solid tumours. Overall survival and YBX1 expression data for cohorts of male and female cancer patients obtained from freely available databases were analysed with a cox proportional hazard model with covariates of biological sex and YBX1 expression. Kaplan–Meier curves and Violin plots were constructed for segregated male and female cohorts. High YBX1 expression was significantly associated with poor survival in 2 female-only and 4 mixed-sex cancer sites. In female lung cancer patients, better survival and lower YBX1 expression were identified. The clinical importance of YBX1 expression in cancer ought to be evaluated in a sex-specific manner, especially in lung cancer.

Objective Cervical screening is a life-saving intervention, which reduces the incidence of and mortality from cervical cancer in the population. Human papillomavirus (HPV) based screening modalities hold unique promise in improving screening accuracy. HPV prevalence varies markedly by age, as does resultant cervical intraepithelial neoplasia (CIN), with higher rates recorded in younger women. With the advent of effective vaccination for HPV drastically reducing prevalence of both HPV and CIN, it is critical to model how the accuracy of different screening approaches varies with age cohort and vaccination status. This work establishes a model for the age-specific prevalence of HPV factoring in vaccine coverage and predicts how the accuracy of common screening modalities is affected by age profile and vaccine uptake.Design Modelling study of HPV infection rates by age, ascertained from European cohorts prior to the introduction of vaccination. Reductions in HPV due to vaccination were estimated from the bounds predicted from multiple modelling studies, yielding a model for age-varying HPV and CIN grades 2 and above (CIN2+) prevalence.Setting Performance of both conventional liquid-based cytology (LBC) screening and HPV screening with LBC reflex (HPV reflex) was estimated under different simulated age cohorts and vaccination levels.Participants Simulated populations of varying age and vaccination status.Results HPV-reflex modalities consistently result in much lower incidence of false positives than LBC testing, with an accuracy that improves even as HPV and CIN2+ rates decline.Conclusions HPV-reflex tests outperform LBC tests across all age profiles, resulting in greater test accuracy. This improvement is especially pronounced as HPV infection rates fall and suggests HPV-reflex modalities are robust to future changes in the epidemiology of HPV.

In biomedical science, it is a reality that many published results do not withstand deeper investigation, and there is growing concern over a replicability crisis in science. Recently, Ellipse of Insignificance (EOI) analysis was introduced as a tool to allow researchers to gauge the robustness of reported results in dichotomous outcome design trials, giving precise deterministic values for the degree of miscoding between events and non-events tolerable simultaneously in both control and experimental arms (Grimes, 2022). While this is useful for situations where potential miscoding might transpire, it does not account for situations where apparently significant findings might result from accidental or deliberate data redaction in either the control or experimental arms of an experiment, or from missing data or systematic redaction. To address these scenarios, we introduce Region of Attainable Redaction (ROAR), a tool that extends EOI analysis to account for situations of potential data redaction. This produces a bounded cubic curve rather than an ellipse, and we outline how this can be used to identify potential redaction through an approach analogous to EOI. Applications are illustrated, and source code, including a web-based implementation that performs EOI and ROAR analysis in tandem for dichotomous outcome trials is provided.

The reality that volumes of published biomedical research are not reproducible is an increasingly recognized problem. Spurious results reduce trustworthiness of reported science, increasing research waste. While science should be self-correcting from a philosophical perspective, that in insolation yields no information on efforts required to nullify suspect findings or factors shaping how quickly science may be corrected. There is also a paucity of information on how perverse incentives in the publishing ecosystem favouring novel positive findings over null results shape the ability of published science to self-correct. Knowledge of factors shaping self-correction of science remain obscure, limiting our ability to mitigate harms. This modelling study introduces a simple model to capture dynamics of the publication ecosystem, exploring factors influencing research waste, trustworthiness, corrective effort and time to correction. Results from this work indicate that research waste and corrective effort are highly dependent on field-specific false positive rates and time delays to corrective results to spurious findings are propagated. The model also suggests conditions under which biomedical science is self-correcting and those under which publication of correctives alone cannot stem propagation of untrustworthy results. Finally, this work models a variety of potential mitigation strategies, including researcher- and publisher-driven interventions.

There is increasing awareness throughout biomedical science that many results do not withstand the trials of repeat investigation. The growing abundance of medical literature has only increased the urgent need for tools to gauge the robustness and trustworthiness of published science. Dichotomous outcome designs are vital in randomized clinical trials, cohort studies, and observational data for ascertaining differences between experimental and control arms. It has however been shown with tools like the fragility index (FI) that many ostensibly impactful results fail to materialize when even small numbers of patients or subjects in either the control or experimental arms are recoded from event to non-event. Critics of this metric counter that there is no objective means to determine a meaningful FI. As currently used, FI is not multidimensional and is computationally expensive. In this work, a conceptually similar geometrical approach is introduced, the ellipse of insignificance. This method yields precise deterministic values for the degree of manipulation or miscoding that can be tolerated simultaneously in both control and experimental arms, allowing for the derivation of objective measures of experimental robustness. More than this, the tool is intimately connected with sensitivity and specificity of the event/non-event tests, and is readily combined with knowledge of test parameters to reject unsound results. The method is outlined here, with illustrative clinical examples.

Background: Few areas of health have been as insidiously influenced by misinformation as cancer. Thus, interventions that can help people impacted by cancer reduce the extent to which they are victims of misinformation are necessary. The Informed Health Choices (IHC) initiative has developed Key Concepts that can be used in the development of interventions for evaluating the trustworthiness of claims about the effects of health treatments. We are developing an online education programme called Informed Health Choices-Cancer (IHC-C) based on the IHC Key Concepts. We will provide those impacted by cancer with the knowledge and skills necessary to think critically about the reliability of health information and claims and make informed choices. Methods: We will establish a steering group (SG) of 12 key stakeholders, including oncology specialists and academics. In addition, we will establish a patient and public involvement (PPI) panel of 20 people impacted by cancer. After training the members on the Key Concepts and the prioritisation process, we will conduct a two-round prioritisation process. In the first round, 12 SG members and four PPI panel members will prioritise Key Concepts for inclusion. In the second round, the remaining 16 PPI members will undertake the prioritisation based on the prioritised Key Concepts from the first round. Participants in both rounds will use a structured judgement form to rate the importance of the Key Concepts for inclusion in the online IHC-C programme. A consensus meeting will be held, where members will reach a consensus on the Key Concepts to be included and rank the order in which the prioritised Key Concepts will be addressed in the IHC-C programme. Conclusions: At the end of this process, we will identify which Key Concepts should be included and the order in which they should be addressed in the IHC-C programme.

A concerning amount of biomedical research is not reproducible. Unreliable results impede empirical progress in medical science, ultimately putting patients at risk. Many proximal causes of this irreproducibility have been identified, a major one being inappropriate statistical methods and analytical choices by investigators. Within this, we formally quantify the impact of inappropriate redaction beyond a threshold value in biomedical science. This is effectively truncation of a dataset by removing extreme data points, and we elucidate its potential to accidentally or deliberately engineer a spurious result in significance testing. We demonstrate that the removal of a surprisingly small number of data points can be used to dramatically alter a result. It is unknown how often redaction bias occurs in the broader literature, but given the risk of distortion to the literature involved, we suggest that it must be studiously avoided, and mitigated with approaches to counteract any potential malign effects to the research quality of medical science.

Despite the wide application of the magnetic resonance imaging (MRI) technique, there are no widely used standards on naming and describing MRI sequences. The absence of consistent naming conventions presents a major challenge in automating image processing since most MRI software require a priori knowledge of the type of the MRI sequences to be processed. This issue becomes increasingly critical with the current efforts toward open-sharing of MRI data in the neuroscience community. This manuscript reports an MRI sequence detection method using imaging metadata and a supervised machine learning technique. Three datasets from the Brain Center for Ontario Data Exploration (Brain-CODE) data platform, each involving MRI data from multiple research institutes, are used to build and test our model. The preliminary results show that a random forest model can be trained to accurately identify MRI sequence types, and to recognize MRI scans that do not belong to any of the known sequence types. Therefore the proposed approach can be used to automate processing of MRI data that involves a large number of variations in sequence names, and to help standardize sequence naming in ongoing data collections. This study highlights the potential of the machine learning approaches in helping manage health data.