Search

Abstract Network meta-analysis compares multiple interventions and estimates the relative treatment effects between all interventions, combining both direct and indirect evidence. Recently, a framework was developed to assess the Risk Of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN) which is part of the more comprehensive framework to evaluate the Confidence In the evidence for Network Meta-Analysis (CINeMA). To produce an overall risk of bias judgement for each network estimate, ROB-MEN: performs an assessment of the bias due to missing evidence in each possible pairwise comparison; combines the assessment with the contribution from the direct pairwise comparisons; considers the potential for small-study effects. To facilitate and semi-automate this process, ROB-MEN has been implemented in a user-friendly web-application ( https://cinema.ispm.unibe.ch/rob-men ). Here we provide a tutorial detailing the functionality and use of the application consisting of data upload, analysis configuration, output visualisation, and production of the tool’s output tables for recording the risk of bias assessment. We also illustrate an example application using the demo dataset available for download on the application’s homepage. The ROB-MEN web-application is open-source and freely available ( https://github.com/esm-ispm-unibe-ch/rob-men ).

Abstract Background Novel survey methods are needed to tackle declining response rates. The 2020 National Maternity Survey included a randomised controlled trial (RCT) and social media survey to compare different combinations of sampling and data collection methods with respect to: response rate, respondent representativeness, prevalence estimates of maternity indicators and cost. Methods A two-armed parallel RCT and concurrent social media survey were conducted. Women in the RCT were sampled from ONS birth registrations and randomised to either a paper or push-to-web survey. Women in the social media survey self-selected through online adverts. The primary outcome was response rate in the paper and push-to-web surveys. In all surveys, respondent representativeness was assessed by comparing distributions of sociodemographic characteristics in respondents with those of the target population. External validity of prevalence estimates of maternity indicators was assessed by comparing weighted survey estimates with estimates from national routine data. Cost was also compared across surveys. Results The response rate was higher in the paper survey (n = 2,446) compared to the push-to-web survey (n = 2,165)(30.6% versus 27.1%, difference = 3.5%, 95%CI = 2.1–4.9, p

Abstract Background Web-surveys are increasingly used in population studies. Yet, web-surveys targeting older individuals are still uncommon for various reasons. However, with younger cohorts approaching older age, the potentials for web-surveys among older people might be improved. In this study, we investigated response patterns in a web-survey targeting older adults and the potential importance of offering a paper-questionnaire as an alternative to the web-questionnaire. Methods We analyzed data from three waves of a retirement study, in which a web-push methodology was used and a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder. We mapped the response patterns, compared web- and paper respondents and compared different key outcomes resulting from the sample with and without the paper respondents, both at baseline and after two follow-ups. Results Paper-respondents, that is, those that did not answer until they got a paper questionnaire with the last reminder, were more likely to be female, retired, single, and to report a lower level of education, higher levels of depression and lower self-reported health, compared to web-respondents. The association between retirement status and depression was only present among web-respondents. The differences between web and paper respondents were stronger in the longitudinal sample (after two follow-ups) than at baseline. Conclusions We conclude that a web-survey might be a feasible and good alternative in surveys targeting people in the retirement age range. However, without offering a paper-questionnaire, a small but important group will likely be missing with potential biased estimates as the result.

Abstract Background The Schedule for the Evaluation of Individual Quality of Life-Direct Weighting (SEIQoL-DW) is a prevalent face-to-face interview method for measuring quality of life by integrating respondent-generated dimensions. To apply this method in clinical trials, a paper-administered alternative would be of interest. Therefore, our study aimed to analyze the agreement between the SEIQoL-DW and a paper questionnaire version (SEIQoL-PF/G). Methods In a crossover design, both measures were completed in a random sequence. 104 patients at a heart surgery hospital in Germany were randomly assigned to receive either the SEIQoL-DW or the SEIQoL-PF/G as the first measurement in the sequence. Patients were approached on their earliest stable day after surgery. The average time between both measurements was 1 day (mean 1.3; SD 0.8). Agreement regarding the indices, ratings, and weightings of nominated life areas (cues) was explored using Bland-Altman plots with 95% limits of agreement (LoA). Agreement of the SEIQoL indices was defined as acceptable if the LoA did not exceed a threshold of 10 scale points. Data from n = 99 patients were included in the agreement analysis. Results Both measures led to similarly nominated cues. The most frequently nominated cues were “physical health” and “family”. In the Bland-Altman plot, the indices showed a mean of differences of 2 points (95% CI, − 1 to 6). The upper LoA showed a difference of 36 points (95% CI, 30 to 42), and the lower LoA showed a difference of − 31 points (95% CI, − 37 to − 26). Thus, the LoAs and confidence intervals exceeded the predefined threshold. The Bland-Altman plots for the cue levels and cue weights showed similar results. The SEIQoL-PF/G version showed a tendency for equal weighting of cues, while the weighting procedure of the SEIQoL-DW led to greater variability. Conclusions For cardiac surgery patients, use of the current version of the SEIQoL-PF/G as a substitute for the SEIQoL-DW is not recommended. The current questionnaire weighting method seems to be unable to distinguish weighting for different cues. Therefore, the further design of a weighting method without interviewer support as a paper-administered measure of individual quality of life is desirable.

Abstract Background Propensity scores are widely used to deal with confounding bias in medical research. An incorrectly specified propensity score model may lead to residual confounding bias; therefore it is essential to use diagnostics to assess propensity scores in a propensity score analysis. The current use of propensity score diagnostics in the medical literature is unknown. The objectives of this study are to (1) assess the use of propensity score diagnostics in medical studies published in high-ranking journals, and (2) assess whether the use of propensity score diagnostics differs between studies (a) in different research areas and (b) using different propensity score methods. Methods A PubMed search identified studies published in high-impact journals between Jan 1st 2014 and Dec 31st 2016 using propensity scores to answer an applied medical question. From each study we extracted information regarding how propensity scores were assessed and which propensity score method was used. Research area was defined using the journal categories from the Journal Citations Report. Results A total of 894 papers were included in the review. Of these, 187 (20.9%) failed to report whether the propensity score had been assessed. Commonly reported diagnostics were p-values from hypothesis tests (36.6%) and the standardised mean difference (34.6%). Statistical tests provided marginally stronger evidence for a difference in diagnostic use between studies in different research areas (p = 0.033) than studies using different propensity score methods (p = 0.061). Conclusions The use of diagnostics in the propensity score medical literature is far from optimal, with different diagnostics preferred in different areas of medicine. The propensity score literature may improve with focused efforts to change practice in areas where suboptimal practice is most common.

Background: Large language models (LLMs) that can efficiently screen and identify studies meeting specific criteria would streamline literature reviews. Additionally, those capable of extracting data from publications would enhance knowledge discovery by reducing the burden on human reviewers. Methods: We created an automated pipeline utilizing OpenAI GPT-4 32 K API version "2023–05-15" to evaluate the accuracy of the LLM GPT-4 responses to queries about published papers on HIV drug resistance (HIVDR) with and without an instruction sheet. The instruction sheet contained specialized knowledge designed to assist a person trying to answer questions about an HIVDR paper. We designed 60 questions pertaining to HIVDR and created markdown versions of 60 published HIVDR papers in PubMed. We presented the 60 papers to GPT-4 in four configurations: (1) all 60 questions simultaneously; (2) all 60 questions simultaneously with the instruction sheet; (3) each of the 60 questions individually; and (4) each of the 60 questions individually with the instruction sheet. Results: GPT-4 achieved a mean accuracy of 86.9% – 24.0% higher than when the answers to papers were permuted. The overall recall and precision were 72.5% and 87.4%, respectively. The standard deviation of three replicates for the 60 questions ranged from 0 to 5.3% with a median of 1.2%. The instruction sheet did not significantly increase GPT-4's accuracy, recall, or precision. GPT-4 was more likely to provide false positive answers when the 60 questions were submitted individually compared to when they were submitted together. Conclusions: GPT-4 reproducibly answered 3600 questions about 60 papers on HIVDR with moderately high accuracy, recall, and precision. The instruction sheet's failure to improve these metrics suggests that more sophisticated approaches are necessary. Either enhanced prompt engineering or finetuning an open-source model could further improve an LLM's ability to answer questions about highly specialized HIVDR papers. [ABSTRACT FROM AUTHOR]

Abstract Background Regulations, study design complexity and amounts of collected and shared data in clinical trials render efficient data handling procedures inevitable. Recent research suggests that electronic data capture can be key in this context but evidence is insufficient. This randomized controlled parallel group study tested the hypothesis that time efficiency is superior when electronic (eCRF) instead of paper case report forms (pCRF) are used for data collection. We additionally investigated predictors of time saving effects and data integrity. Methods This study was conducted on top of a clinical weight loss trial performed at a clinical research facility over six months. All study nurses and patients participating in the clinical trial were eligible to participate and randomly allocated to enter cross-sectional data obtained during routine visits either through pCRF or eCRF. A balanced randomization list was generated before enrolment commenced. 90 and 30 records were gathered for the time that 27 patients and 2 study nurses required to report 2025 and 2037 field values, respectively. The primary hypothesis, that eCRF use is faster than pCRF use, was tested by a two-tailed t-test. Analysis of variance and covariance were used to evaluate predictors of entry performance. Data integrity was evaluated by descriptive statistics. Results All randomized patients were included in the study (eCRF group n = 13, pCRF group n = 14). eCRF, as compared to pCRF, data collection was associated with significant time savings across all conditions (8.29 ± 5.15 min vs. 10.54 ± 6.98 min, p = .047). This effect was not defined by participant type, i.e. patients or study nurses (F(1,112) = .15, p = .699), CRF length (F(2,112) = .49, p = .609) or patient age (Beta = .09, p = .534). Additional 5.16 ± 2.83 min per CRF were saved with eCRFs due to data transcription redundancy when patients answered questionnaires directly in eCRFs. Data integrity was superior in the eCRF condition (0 versus 3 data entry errors). Conclusions This is the first study to prove in direct comparison that using eCRFs instead of pCRFs increases time efficiency of data collection in clinical trials, irrespective of item quantity or patient age, and improves data quality. Trial registration Clinical Trials NCT02649907 .

Background: Previous research has found that a "web-push" approach to data collection, which involves contacting people by mail to request an Internet survey response while withholding a paper response option until later in the contact process, consistently achieves lower response rates than a "paper-only" approach, whereby all respondents are contacted and requested to respond by mail., Method: An experiment was designed, as part of the Millennium Cohort Family Study, to compare response rates, sample representativeness, and cost between a web-push and a paper-only approach; each approach comprised 3 stages of mail contacts. The invited sample (n = 4,935) consisted of spouses married to U.S. Service members, who had been serving in the military between 2 and 5 years as of October, 2011., Results: The web-push methodology produced a significantly higher response rate, 32.8% compared to 27.8%. Each of the 3 stages of postal contact significantly contributed to response for both treatments with 87.1% of the web-push responses received over the Internet. The per-respondent cost of the paper-only treatment was almost 40% higher than the web-push treatment group. Analyses revealed no meaningfully significant differences between treatment groups in representation., Conclusion: These results provide evidence that a web-push methodology is more effective and less expensive than a paper-only approach among young military spouses, perhaps due to their heavy reliance on the internet, and we suggest that this approach may be more effective with the general population as they become more uniformly internet savvy.

Background: With advances in cancer diagnosis and treatment, women with early-stage breast cancer (ESBC) are living longer, increasing the number of patients receiving post-treatment follow-up care. Best-practice survivorship models recommend transitioning ESBC patients from oncology-provider (OP) care to community-based care. While developing materials for a future randomized controlled trial (RCT) to test the feasibility of a nurse-led Telephone Survivorship Clinic (TSC) for a smooth transition of ESBC survivors to follow-up care, we explored patients' and OPs' reactions to several of our proposed methods.Methods: We used a qualitative study design with thematic analysis and a two-pronged approach. We interviewed OPs, seeking feedback on ways to recruit their ESBC patients for the trial, and ESBC patients, seeking input on a questionnaire package assessing outcomes and processes in the trial.Results: OPs identified facilitators and barriers and offered suggestions for study design and recruitment process improvement. Facilitators included the novelty and utility of the study and simplicity of methods; barriers included lack of coordination between treating and discharging clinicians, time constraints, language barriers, motivation, and using a paper-based referral letter. OPs suggested using a combination of electronic and paper referral letters and supporting clinicians to help with recruitment. Patient advisors reported satisfaction with the content and length of the assessment package. However, they questioned the relevance of some questions (childhood trauma) while adding questions about trust in physicians and proximity to primary-care providers.Conclusions: OPs and patient advisors rated our methods for the proposed trial highly for their simplicity and relevance then suggested changes. These findings document processes that could be effective for cancer-patient recruitment in survivorship clinical trials. [ABSTRACT FROM AUTHOR]

Introduction: Non-compliance is a common challenge for researchers and may reduce the power of an intention-to-treat analysis. Whilst a per protocol approach attempts to deal with this issue, it can result in biased estimates. Several methods to resolve this issue have been identified in previous reviews, but there is limited evidence supporting their use. This review aimed to identify simulation studies which compare such methods, assess the extent to which certain methods have been investigated and determine their performance under various scenarios. Methods: A systematic search of several electronic databases including MEDLINE and Scopus was carried out from conception to 30th November 2022. Included papers were published in a peer-reviewed journal, readily available in the English language and focused on comparing relevant methods in a superiority randomised controlled trial under a simulation study. Articles were screened using these criteria and a predetermined extraction form used to identify relevant information. A quality assessment appraised the risk of bias in individual studies. Extracted data was synthesised using tables, figures and a narrative summary. Both screening and data extraction were performed by two independent reviewers with disagreements resolved by consensus. Results: Of 2325 papers identified, 267 full texts were screened and 17 studies finally included. Twelve methods were identified across papers. Instrumental variable methods were commonly considered, but many authors found them to be biased in some settings. Non-compliance was generally assumed to be all-or-nothing and only occurring in the intervention group, although some methods considered it as time-varying. Simulation studies commonly varied the level and type of non-compliance and factors such as effect size and strength of confounding. The quality of papers was generally good, although some lacked detail and justification. Therefore, their conclusions were deemed to be less reliable. Conclusions: It is common for papers to consider instrumental variable methods but more studies are needed that consider G-methods and compare a wide range of methods in realistic scenarios. It is difficult to make conclusions about the best method to deal with non-compliance due to a limited body of evidence and the difficulty in combining results from independent simulation studies. PROSPERO registration number: CRD42022370910. [ABSTRACT FROM AUTHOR]

Background: Response rates to surveys are declining and this threatens the validity and generalisability of their findings. We wanted to determine whether paper quality influences the response rate to postal surveys, Methods: A postal questionnaire was sent to all members of the British Society of Gynaecological Endoscopy (BSGE). Recipients were randomised to receiving the questionnaire printed on standard quality paper or high quality paper., Results: The response rate for the recipients of high quality paper was 43/195 (22%) and 57/194 (29%) for standard quality paper (relative rate of response 0.75, 95% CI 0.33-1.05, p = 0.1, Conclusion: The use of high quality paper did not increase response rates to a questionnaire survey of gynaecologists affiliated to an endoscopic society.

Abstract Background The preliminary results of a study are usually presented as an abstract in conference meetings. The reporting quality of those abstracts and the relationship between their study designs and full paper publication rate is unknown. We hypothesized that randomized controlled trials are more likely to be published as full papers than observational studies. Methods 154 oral abstracts presented at the World Congress of Sports Injury Prevention 2005 Oslo and the corresponding full paper publication were identified and analysed. The main outcome measures were frequency of publication, time to publication, impact factor, CONSORT (for Consolidated Standards of Reporting Trials) score, STROBE (for Strengthening the Reporting of Observational Studies in Epidemiology) score, and minor and major inconsistencies between the abstract and the full paper publication. Results Overall, 76 of the 154 (49%) presented abstracts were published as full papers in a peer-reviewed journal with an impact factor of 1.946 ± 0.812. No significant difference existed between the impact factor for randomized controlled trials (2.122 ± 1.015) and observational studies (1.913 ± 0.765, p = 0.469). The full papers for the randomized controlled trials were published after an average (SD) of 17 months (± 13 months); for observational studies, the average (SD) was 12 months (± 14 months) (p = 0.323). A trend was observed in this study that a higher percentage of randomized controlled trial abstracts were published as full papers (71% vs. 47%, p = 0.078) than observational trials. The reporting quality of abstracts, published as full papers, significantly increased compared to conference abstracts both in randomized control studies (CONSORT: 5.7 ± 0.7 to 7.2 ± 1.3; p = 0.018, CI -2.7 to -0.32) and in observational studies (STROBE: 8.2 ± 1.3 to 8.6 ± 1.4; p = 0.007, CI -0.63 to -0.10). All of the published abstracts had at least one minor inconsistency (title, authors, research center, outcome presentation, conclusion), while 65% had at least major inconsistencies (study objective, hypothesis, study design, primary outcome measures, sample size, statistical analysis, results, SD/CI). Comparing the results of conference and full paper; results changed in 90% vs. 68% (randomized, controlled studies versus observational studies); data were added (full paper reported more result data) in 60% vs. 30%, and deleted (full paper reported fewer result data) in 40% vs. 30%. Conclusions No significant differences with respect to type of study (randomized controlled versus observational), impact factor, and time to publication existed for the likelihood that a World Congress of Sports Injury conference abstract could be published as a full paper.

Background: Regulations, study design complexity and amounts of collected and shared data in clinical trials render efficient data handling procedures inevitable. Recent research suggests that electronic data capture can be key in this context but evidence is insufficient. This randomized controlled parallel group study tested the hypothesis that time efficiency is superior when electronic (eCRF) instead of paper case report forms (pCRF) are used for data collection. We additionally investigated predictors of time saving effects and data integrity., Methods: This study was conducted on top of a clinical weight loss trial performed at a clinical research facility over six months. All study nurses and patients participating in the clinical trial were eligible to participate and randomly allocated to enter cross-sectional data obtained during routine visits either through pCRF or eCRF. A balanced randomization list was generated before enrolment commenced. 90 and 30 records were gathered for the time that 27 patients and 2 study nurses required to report 2025 and 2037 field values, respectively. The primary hypothesis, that eCRF use is faster than pCRF use, was tested by a two-tailed t-test. Analysis of variance and covariance were used to evaluate predictors of entry performance. Data integrity was evaluated by descriptive statistics., Results: All randomized patients were included in the study (eCRF group n = 13, pCRF group n = 14). eCRF, as compared to pCRF, data collection was associated with significant time savings  across all conditions (8.29 ± 5.15 min vs. 10.54 ± 6.98 min, p = .047). This effect was not defined by participant type, i.e. patients or study nurses (F (1,112)  = .15, p = .699), CRF length (F (2,112)  = .49, p = .609) or patient age (Beta = .09, p = .534). Additional 5.16 ± 2.83 min per CRF were saved with eCRFs due to data transcription redundancy when patients answered questionnaires directly in eCRFs. Data integrity was superior in the eCRF condition (0 versus 3 data entry errors)., Conclusions: This is the first study to prove in direct comparison that using eCRFs instead of pCRFs increases time efficiency of data collection in clinical trials, irrespective of item quantity or patient age, and improves data quality., Trial Registration: Clinical Trials NCT02649907 .

Abstract Background Computerized adaptive testing (CAT) is being applied to health outcome measures developed as paper-and-pencil (P&P) instruments. Differences in how respondents answer items administered by CAT vs. P&P can increase error in CAT-estimated measures if not identified and corrected. Method Two methods for detecting item-level mode effects are proposed using Bayesian estimation of posterior distributions of item parameters: (1) a modified robust Z (RZ) test, and (2) 95% credible intervals (CrI) for the CAT-P&P difference in item difficulty. A simulation study was conducted under the following conditions: (1) data-generating model (one- vs. two-parameter IRT model); (2) moderate vs. large DIF sizes; (3) percentage of DIF items (10% vs. 30%), and (4) mean difference in θ estimates across modes of 0 vs. 1 logits. This resulted in a total of 16 conditions with 10 generated datasets per condition. Results Both methods evidenced good to excellent false positive control, with RZ providing better control of false positives and with slightly higher power for CrI, irrespective of measurement model. False positives increased when items were very easy to endorse and when there with mode differences in mean trait level. True positives were predicted by CAT item usage, absolute item difficulty and item discrimination. RZ outperformed CrI, due to better control of false positive DIF. Conclusions Whereas false positives were well controlled, particularly for RZ, power to detect DIF was suboptimal. Research is needed to examine the robustness of these methods under varying prior assumptions concerning the distribution of item and person parameters and when data fail to conform to prior assumptions. False identification of DIF when items were very easy to endorse is a problem warranting additional investigation.

Abstract Background Recruiting to primary care studies is complex. With the current drive to increase numbers of patients involved in primary care studies, we need to know more about successful recruitment approaches. There is limited evidence on recruitment to focus group studies, particularly when no natural grouping exists and where participants do not regularly meet. The aim of this paper is to reflect on recruitment to a focus group study comparing the methods used with existing evidence using a resource for research recruitment, PROSPeR (Planning Recruitment Options: Strategies for Primary Care). Methods The focus group formed part of modelling a complex intervention in primary care in the Resources for Effective Sleep Treatment (REST) study. Despite a considered approach at the design stage, there were a number of difficulties with recruitment. The recruitment strategy and subsequent revisions are detailed. Results The researchers' modifications to recruitment, justifications and evidence from the literature in support of them are presented. Contrary evidence is used to analyse why some aspects were unsuccessful and evidence is used to suggest improvements. Recruitment to focus group studies should be considered in two distinct phases; getting potential participants to contact the researcher, and converting those contacts into attendance. The difficulty of recruitment in primary care is underemphasised in the literature especially where people do not regularly come together, typified by this case study of patients with sleep problems. Conclusion We recommend training GPs and nurses to recruit patients during consultations. Multiple recruitment methods should be employed from the outset and the need to build topic related non-financial incentives into the group meeting should be considered. Recruitment should be monitored regularly with barriers addressed iteratively as a study progresses.

Abstract Background Given an observed test statistic and its degrees of freedom, one may compute the observed P value with most statistical packages. It is unknown to what extent test statistics and P values are congruent in published medical papers. Methods We checked the congruence of statistical results reported in all the papers of volumes 409–412 of Nature (2001) and a random sample of 63 results from volumes 322–323 of BMJ (2001). We also tested whether the frequencies of the last digit of a sample of 610 test statistics deviated from a uniform distribution (i.e., equally probable digits). Results 11.6% (21 of 181) and 11.1% (7 of 63) of the statistical results published in Nature and BMJ respectively during 2001 were incongruent, probably mostly due to rounding, transcription, or type-setting errors. At least one such error appeared in 38% and 25% of the papers of Nature and BMJ, respectively. In 12% of the cases, the significance level might change one or more orders of magnitude. The frequencies of the last digit of statistics deviated from the uniform distribution and suggested digit preference in rounding and reporting. Conclusions This incongruence of test statistics and P values is another example that statistical practice is generally poor, even in the most renowned scientific journals, and that quality of papers should be more controlled and valued.

Abstract Background Response rates to surveys are declining and this threatens the validity and generalisability of their findings. We wanted to determine whether paper quality influences the response rate to postal surveys Methods A postal questionnaire was sent to all members of the British Society of Gynaecological Endoscopy (BSGE). Recipients were randomised to receiving the questionnaire printed on standard quality paper or high quality paper. Results The response rate for the recipients of high quality paper was 43/195 (22%) and 57/194 (29%) for standard quality paper (relative rate of response 0.75, 95% CI 0.33–1.05, p = 0.1 Conclusion The use of high quality paper did not increase response rates to a questionnaire survey of gynaecologists affiliated to an endoscopic society.

The article focuses on a study related to sports injuries. About 154 oral abstracts presented at the World Congress of Sports Injury Prevention 2005 Oslo and the corresponding full paper publication were identified and analyzed. A considerable proportion of health care research are first presented at conferences and meetings, and published as abstracts in the proceedings.

Background Electronic Case Report Forms (eCRFs) are increasingly chosen by investigators and sponsors of clinical research instead of the traditional pen-and-paper data collection (pCRFs). Previous studies suggested that eCRFs avoided mistakes, shortened the duration of clinical studies and reduced data collection costs. Methods Our objectives were to describe and contrast both objective and subjective efficiency of pCRF and eCRF use in clinical studies. A total of 27 studies (11 eCRF, 16 pCRF) sponsored by the Paris hospital consortium, conducted and completed between 2001 and 2011 were included. Questionnaires were emailed to investigators of those studies, as well as clinical research associates and data managers working in Paris hospitals, soliciting their level of satisfaction and preferences for eCRFs and pCRFs. Mean costs and timeframes were compared using bootstrap methods, linear and logistic regression. Results The total cost per patient was 374 € ±351 with eCEFs vs. 1,135 € ± ±351 with eCEFs between the opening of the first center and the database lock was 31.7 months Q1 = 24.6; Q3 = 42.8 using eCRFs, vs. 39.8 months Q1 = 31.7; Q3 = 52.2 with pCRFs (p = 0.11). Electronic CRFs were globally preferred by all (31/72 vs. 15/72 for paper) for easier monitoring and improved data quality. Conclusions This study found that eCRFs and pCRFs are used in studies with different patient numbers, center numbers and risk. The first ones are more advantageous in large, low-risk studies and gain support from a majority of stakeholders. [ABSTRACT FROM AUTHOR]

Background: Health and social care interventions are often complex and can be decomposed into multiple components. Multicomponent interventions are often evaluated in randomised controlled trials. Across trials, interventions often have components in common which are given alongside other components which differ across trials. Multicomponent interventions can be synthesised using component NMA (CNMA). CNMA is limited by the structure of the available evidence, but it is not always straightforward to visualise such complex evidence networks. The aim of this paper is to develop tools to visualise the structure of complex evidence networks to support CNMA. Methods: We performed a citation review of two key CNMA methods papers to identify existing published CNMA analyses and reviewed how they graphically represent intervention complexity and comparisons across trials. Building on identified shortcomings of existing visualisation approaches, we propose three approaches to standardise visualising the data structure and/or availability of data: CNMA-UpSet plot, CNMA heat map, CNMA-circle plot. We use a motivating example to illustrate these plots. Results: We identified 34 articles reporting CNMAs. A network diagram was the most common plot type used to visualise the data structure for CNMA (26/34 papers), but was unable to express the complex data structures and large number of components and potential combinations of components associated with CNMA. Therefore, we focused visualisation development around representing the data structure of a CNMA more completely. The CNMA-UpSet plot presents arm-level data and is suitable for networks with large numbers of components or combinations of components. Heat maps can be utilised to inform decisions about which pairwise interactions to consider for inclusion in a CNMA model. The CNMA-circle plot visualises the combinations of components which differ between trial arms and offers flexibility in presenting additional information such as the number of patients experiencing the outcome of interest in each arm. Conclusions: As CNMA becomes more widely used for the evaluation of multicomponent interventions, the novel CNMA-specific visualisations presented in this paper, which improve on the limitations of existing visualisations, will be important to aid understanding of the complex data structure and facilitate interpretation of the CNMA results. [ABSTRACT FROM AUTHOR]

Background: Electronic Case Report Forms (eCRFs) are increasingly chosen by investigators and sponsors of clinical research instead of the traditional pen-and-paper data collection (pCRFs). Previous studies suggested that eCRFs avoided mistakes, shortened the duration of clinical studies and reduced data collection costs., Methods: Our objectives were to describe and contrast both objective and subjective efficiency of pCRF and eCRF use in clinical studies. A total of 27 studies (11 eCRF, 16 pCRF) sponsored by the Paris hospital consortium, conducted and completed between 2001 and 2011 were included. Questionnaires were emailed to investigators of those studies, as well as clinical research associates and data managers working in Paris hospitals, soliciting their level of satisfaction and preferences for eCRFs and pCRFs. Mean costs and timeframes were compared using bootstrap methods, linear and logistic regression., Results: The total cost per patient was 374€ ±351 with eCRFs vs. 1,135€ ±1,234 with pCRFs. Time between the opening of the first center and the database lock was 31.7 months Q1 = 24.6; Q3 = 42.8 using eCRFs, vs. 39.8 months Q1 = 31.7; Q3 = 52.2 with pCRFs (p = 0.11). Electronic CRFs were globally preferred by all (31/72 vs. 15/72 for paper) for easier monitoring and improved data quality., Conclusions: This study found that eCRFs and pCRFs are used in studies with different patient numbers, center numbers and risk. The first ones are more advantageous in large, low-risk studies and gain support from a majority of stakeholders.

Background: Given an observed test statistic and its degrees of freedom, one may compute the observed P value with most statistical packages. It is unknown to what extent test statistics and P values are congruent in published medical papers. Methods: We checked the congruence of statistical results reported in all the papers of volumes 409-412 of Nature (2001) and a random sample of 63 results from volumes 322-323 of BMJ (2001). We also tested whether the frequencies of the last digit of a sample of 610 test statistics deviated from a uniform distribution (i.e., equally probable digits). Results: 11.6% (21 of 181) and 11.1% (7 of 63) of the statistical results published in Nature and BMJ respectively during 2001 were incongruent, probably mostly due to rounding, transcription, or type-setting errors. At least one such error appeared in 38% and 25% of the papers of Nature and BMJ, respectively. In 12% of the cases, the significance level might change one or more orders of magnitude. The frequencies of the last digit of statistics deviated from the uniform distribution and suggested digit preference in rounding and reporting. Conclusions: This incongruence of test statistics and P values is another example that statistical practice is generally poor, even in the most renowned scientific journals, and that quality of papers should be more controlled and valued. [ABSTRACT FROM AUTHOR]

Background: Recruiting to primary care studies is complex. With the current drive to increase numbers of patients involved in primary care studies, we need to know more about successful recruitment approaches. There is limited evidence on recruitment to focus group studies, particularly when no natural grouping exists and where participants do not regularly meet. The aim of this paper is to reflect on recruitment to a focus group study comparing the methods used with existing evidence using a resource for research recruitment, PROSPeR (Planning Recruitment Options: Strategies for Primary Care)., Methods: The focus group formed part of modelling a complex intervention in primary care in the Resources for Effective Sleep Treatment (REST) study. Despite a considered approach at the design stage, there were a number of difficulties with recruitment. The recruitment strategy and subsequent revisions are detailed., Results: The researchers' modifications to recruitment, justifications and evidence from the literature in support of them are presented. Contrary evidence is used to analyse why some aspects were unsuccessful and evidence is used to suggest improvements. Recruitment to focus group studies should be considered in two distinct phases; getting potential participants to contact the researcher, and converting those contacts into attendance. The difficulty of recruitment in primary care is underemphasised in the literature especially where people do not regularly come together, typified by this case study of patients with sleep problems., Conclusion: We recommend training GPs and nurses to recruit patients during consultations. Multiple recruitment methods should be employed from the outset and the need to build topic related non-financial incentives into the group meeting should be considered. Recruitment should be monitored regularly with barriers addressed iteratively as a study progresses.

Background: As evidence related to the COVID-19 pandemic surged, databases, platforms, and repositories evolved with features and functions to assist users in promptly finding the most relevant evidence. In response, research synthesis teams adopted novel searching strategies to sift through the vast amount of evidence to synthesize and disseminate the most up-to-date evidence. This paper explores the key database features that facilitated systematic searching for rapid evidence synthesis during the COVID-19 pandemic to inform knowledge management infrastructure during future global health emergencies. Methods: This paper outlines the features and functions of previously existing and newly created evidence sources routinely searched as part of the NCCMT's Rapid Evidence Service methods, including databases, platforms, and repositories. Specific functions of each evidence source were assessed as they pertain to searching in the context of a public health emergency, including the topics of indexed citations, the level of evidence of indexed citations, and specific usability features of each evidence source. Results: Thirteen evidence sources were assessed, of which four were newly created and nine were either pre-existing or adapted from previously existing resources. Evidence sources varied in topics indexed, level of evidence indexed, and specific searching functions. Conclusion: This paper offers insights into which features enabled systematic searching for the completion of rapid reviews to inform decision makers within 5–10 days. These findings provide guidance for knowledge management strategies and evidence infrastructures during future public health emergencies. [ABSTRACT FROM AUTHOR]

Background Syntheses of qualitative studies can inform health policy, services and our understanding of patient experience. Meta-ethnography is a systematic seven-phase interpretive qualitative synthesis approach well-suited to producing new theories and conceptual models. However, there are concerns about the quality of meta-ethnography reporting, particularly the analysis and synthesis processes. Our aim was to investigate the application and reporting of methods in recent meta-ethnography journal papers, focusing on the analysis and synthesis process and output. Methods Methodological systematic review of health-related meta-ethnography journal papers published from 2012-2013. We searched six electronic databases, Google Scholar and Zetoc for papers using key terms including 'meta-ethnography.' Two authors independently screened papers by title and abstract with 100% agreement. We identified 32 relevant papers. Three authors independently extracted data and all authors analysed the application and reporting of methods using content analysis. Results Meta-ethnography was applied in diverse ways, sometimes inappropriately. In 13% of papers the approach did not suit the research aim. In 66% of papers reviewers did not follow the principles of meta-ethnography. The analytical and synthesis processes were poorly reported overall. In only 31% of papers reviewers clearly described how they analysed conceptual data from primary studies (phase 5, 'translation' of studies) and in only one paper (3%) reviewers explicitly described how they conducted the analytic synthesis process (phase 6). In 38% of papers we could not ascertain if reviewers had achieved any new interpretation of primary studies. In over 30% of papers seminal methodological texts which could have informed methods were not cited. Conclusions We believe this is the first in-depth methodological systematic review of meta-ethnography conduct and reporting. Meta-ethnography is an evolving approach. Current reporting of methods, analysis and synthesis lacks clarity and comprehensiveness. This is a major barrier to use of meta-ethnography findings that could contribute significantly to the evidence base because it makes judging their rigour and credibility difficult. To realise the high potential value of meta-ethnography for enhancing health care and understanding patient experience requires reporting that clearly conveys the methodology, analysis and findings. Tailored meta-ethnography reporting guidelines, developed through expert consensus, could improve reporting. [ABSTRACT FROM AUTHOR]

Background: Health policy decisions are often informed by estimates of long-term survival based primarily on short-term data. A range of methods are available to include longer-term information, but there has previously been no comprehensive and accessible tool for implementing these. Results: This paper introduces a novel model and software package for parametric survival modelling of individual-level, right-censored data, optionally combined with summary survival data on one or more time periods. It could be used to estimate long-term survival based on short-term data from a clinical trial, combined with longer-term disease registry or population data, or elicited judgements. All data sources are represented jointly in a Bayesian model. The hazard is modelled as an M-spline function, which can represent potential changes in the hazard trajectory at any time. Through Bayesian estimation, the model automatically adapts to fit the available data, and acknowledges uncertainty where the data are weak. Therefore long-term estimates are only confident if there are strong long-term data, and inferences do not rely on extrapolating parametric functions learned from short-term data. The effects of treatment or other explanatory variables can be estimated through proportional hazards or with a flexible non-proportional hazards model. Some commonly-used mechanisms for survival can also be assumed: cure models, additive hazards models with known background mortality, and models where the effect of a treatment wanes over time. All of these features are provided for the first time in an R package, survextrap, in which models can be fitted using standard R survival modelling syntax. This paper explains the model, and demonstrates the use of the package to fit a range of models to common forms of survival data used in health technology assessments. Conclusions: This paper has provided a tool that makes comprehensive and principled methods for survival extrapolation easily usable. [ABSTRACT FROM AUTHOR]

Abstract Background Self-interviews, where the respondent rather than the interviewer enters answers to questions, have been proposed as a way to reduce social desirability bias associated with interviewer-led interviews. Computer-assisted self-interviews (CASI) are commonly proposed since the computer programme can guide respondents; however they require both language and computer literacy. We evaluated the feasibility and acceptability of using electronic methods to administer quantitative sexual behaviour questionnaires in the Somkhele demographic surveillance area (DSA) in rural KwaZulu-Natal, South Africa. Methods We conducted a four-arm randomized trial of paper-and-pen-interview, computer-assisted personal-interview (CAPI), CASI and audio-CASI with an age-sex-urbanicity stratified sample of 504 adults resident in the DSA in 2015. We compared respondents’ answers to their responses to the same questions in previous surveillance rounds. We also conducted 48 cognitive interviews, dual-coding responses using the Framework approach. Results Three hundred forty (67%) individuals were interviewed and covariates and participation rates were balanced across arms. CASI and audio-CASI were significantly slower than interviewer-led interviews. Item non-response rates were higher in self-interview arms. In single-paper meta-analysis, self-interviewed individuals reported more socially undesirable sexual behaviours. Cognitive interviews found high acceptance of both self-interviews and the use of electronic methods, with some concerns that self-interview methods required more participant effort and literacy. Conclusions Electronic data collection methods, including self-interview methods, proved feasible and acceptable for completing quantitative sexual behaviour questionnaires in a poor, rural South African setting. However, each method had both benefits and costs, and the choice of method should be based on context-specific criteria.

When different researchers study the same research question using the same dataset they may obtain different and potentially even conflicting results. This is because there is often substantial flexibility in researchers' analytical choices, an issue also referred to as "researcher degrees of freedom". Combined with selective reporting of the smallest p-value or largest effect, researcher degrees of freedom may lead to an increased rate of false positive and overoptimistic results. In this paper, we address this issue by formalizing the multiplicity of analysis strategies as a multiple testing problem. As the test statistics of different analysis strategies are usually highly dependent, a naive approach such as the Bonferroni correction is inappropriate because it leads to an unacceptable loss of power. Instead, we propose using the "minP" adjustment method, which takes potential test dependencies into account and approximates the underlying null distribution of the minimal p-value through a permutation-based procedure. This procedure is known to achieve more power than simpler approaches while ensuring a weak control of the family-wise error rate. We illustrate our approach for addressing researcher degrees of freedom by applying it to a study on the impact of perioperative p a O 2 on post-operative complications after neurosurgery. A total of 48 analysis strategies are considered and adjusted using the minP procedure. This approach allows to selectively report the result of the analysis strategy yielding the most convincing evidence, while controlling the type 1 error—and thus the risk of publishing false positive results that may not be replicable. [ABSTRACT FROM AUTHOR]

Background: Patterns of survey response and the characteristics associated with response over time in longitudinal studies are important to discern for the development of tailored retention efforts aimed at minimizing response bias. The Millennium Cohort Study, the largest and longest running cohort study of military personnel and veterans, is designed to examine the long-term health effects of military service and experiences and thus relies on continued participant survey responses over time. Here, we describe the response rates for follow-up survey data collected over 15 years and identify characteristics associated with follow-up survey response and mode of response (paper vs. web). Method: Patterns of follow-up survey response and response mode (web, paper, none) were examined among eligible participants (n=198,833), who were initially recruited in four panels from 2001 to 2013 in the Millennium Cohort Study, for a follow-up period of 3–15 years (2004–2016). Military and sociodemographic factors (i.e., enrollment panel, sex, birth year, race and ethnicity, educational attainment, marital status, service component, service branch, pay grade, military occupation, length of service, and time deployed), life experiences and health-related factors (i.e., military deployment/combat experience, life stressors, mental health, physical health, and unhealthy behaviors) were used to examine follow-up response and survey mode over time in multivariable generalized estimating equation models. Results: Overall, an average response rate of 60% was observed across all follow-up waves. Factors associated with follow-up survey response over time included increased educational attainment, married status, female sex, older age, military deployment (regardless of combat experience), and higher number of life stressors, mental health issues, and physical health diagnoses. Conclusion: Despite the challenges associated with collecting multiple waves of follow-up survey data from members of the U.S. military during and after service, the Millennium Cohort Study has maintained a relatively robust response rate over time. The incorporation of tailored messages and outreach to those groups least likely to respond over time may improve retention and thereby increase the representativeness and generalizability of collected survey data. [ABSTRACT FROM AUTHOR]

Background: The application of adaptive design methodology within a clinical trial setting is becoming increasingly popular. However the application of these methods within trials is not being reported as adaptive designs hence making it more difficult to capture the emerging use of these designs. Within this review, we aim to understand how adaptive design methodology is being reported, whether these methods are explicitly stated as an 'adaptive design' or if it has to be inferred and to identify whether these methods are applied prospectively or concurrently.Methods: Three databases; Embase, Ovid and PubMed were chosen to conduct the literature search. The inclusion criteria for the review were phase II, phase III and phase II/III randomised controlled trials within the field of Oncology that published trial results in 2015. A variety of search terms related to adaptive designs were used.Results: A total of 734 results were identified, after screening 54 were eligible. Adaptive designs were more commonly applied in phase III confirmatory trials. The majority of the papers performed an interim analysis, which included some sort of stopping criteria. Additionally only two papers explicitly stated the term 'adaptive design' and therefore for most of the papers, it had to be inferred that adaptive methods was applied. Sixty-five applications of adaptive design methods were applied, from which the most common method was an adaptation using group sequential methods.Conclusions: This review indicated that the reporting of adaptive design methodology within clinical trials needs improving. The proposed extension to the current CONSORT 2010 guidelines could help capture adaptive design methods. Furthermore provide an essential aid to those involved with clinical trials. [ABSTRACT FROM AUTHOR]

Aims: Standard outcome sets enable the value-based evaluation of health care delivery. Whereas the attainment of expert opinion has been structured using methods such as the modified-Delphi process, standardized guidelines for extraction of candidate outcomes from literature are lacking. As such, we aimed to describe an approach to obtain a comprehensive list of candidate outcomes for potential inclusion in standard outcome sets. Methods: This study describes an iterative saturation approach, using randomly selected batches from a systematic literature search to develop a long list of candidate outcomes to evaluate healthcare. This approach can be preceded with an optional benchmark review of relevant registries and Clinical Practice Guidelines and data visualization techniques (e.g. as a WordCloud) to potentially decrease the number of iterations. The development of the International Consortium of Health Outcome Measures Heart valve disease set is used to illustrate the approach. Batch cutoff choices of the iterative saturation approach were validated using data of 1000 simulated cases. Results: Simulation showed that on average 98% (range 92–100%) saturation is reached using a 100-article batch initially, with 25 articles in the subsequent batches. On average 4.7 repeating rounds (range 1–9) of 25 new articles were necessary to achieve saturation if no outcomes are first identified from a benchmark review or a data visualization. Conclusion: In this paper a standardized approach is proposed to identify relevant candidate outcomes for a standard outcome set. This approach creates a balance between comprehensiveness and feasibility in conducting literature reviews for the identification of candidate outcomes. [ABSTRACT FROM AUTHOR]

Background: In the data pipeline from the data collection process to the planned statistical analyses, initial data analysis (IDA) typically takes place between the end of the data collection and do not touch the research questions. A systematic process for IDA and clear reporting of the findings would help to understand the potential shortcomings of a dataset, such as missing values, or subgroups with small sample sizes, or shortcomings in the collection process, and to evaluate the impact of these shortcomings on the research results. A clear reporting of findings is also relevant when making datasets available to other researchers. Initial data analyses can provide valuable insights into the suitability of a data set for a future research study. Our aim was to describe the practice of reporting of initial data analyses in observational studies in five highly ranked medical journals with focus on data cleaning, screening, and reporting of findings which led to a potential change in the analysis plan.Methods: This review was carried out using systematic search strategies with eligibility criteria for articles to be reviewed. A total of 25 papers about observational studies were selected from five medical journals published in 2018. Each paper was reviewed by two reviewers and IDA statements were further discussed by all authors. The consensus was reported.Results: IDA statements were reported in the methods, results, discussion, and supplement of papers. Ten out of 25 papers (40%) included a statement about data cleaning. Data screening statements were included in all articles, and 18 (72%) indicated the methods used to describe them. Item missingness was reported in 11 papers (44%), unit missingness in 15 papers (60%). Eleven papers (44%) mentioned some changes in the analysis plan. Reported changes referred to missing data treatment, unexpected values, population heterogeneity and aspects related to variable distributions or data properties.Conclusion: Reporting of initial data analyses were sparse, and statements on IDA were located throughout the research articles. There is a lack of systematic reporting of IDA. We conclude the article with recommendations on how to overcome shortcomings in the practice of IDA reporting in observational studies. [ABSTRACT FROM AUTHOR]

Background: This paper is part of a broader investigation into the ways in which health and social care guideline producers are using qualitative evidence syntheses (QESs) alongside more established methods of guideline development such as systematic reviews and meta-analyses of quantitative data. This study is a content analysis of QESs produced over a 5-year period by a leading provider of guidelines for the National Health Service in the UK (the National Institute for Health and Care Excellence) to explore how closely they match a reporting framework for QES.Methods: Guidelines published or updated between Jan 2015 and Dec 2019 were identified via searches of the National Institute for Health and Care excellence (NICE) website. These guidelines were searched to identify any QES conducted during the development of the guideline. Data relating to the compliance of these syntheses against a reporting framework for QES (ENTREQ) were extracted and compiled, and descriptive statistics used to provide an analysis of the of QES conduct, reporting and use by this major international guideline producer.Results: QES contributed, in part, to 54 out of a total of 192 guidelines over the five-year period. Although methods for producing and reporting QES have changed substantially over the past decade, this study found that there has been little change in the number or quality of NICE QESs over time. The largest predictor of quality was the centre or team which undertook the synthesis. Analysis indicated that elements of review methods which were similar to those used in quantitative systematic reviews tended to be carried out well and mostly matched the criteria in the reporting framework, but review methods which were more specific to a QES tended to be carried out less well, with fewer examples of criteria in the reporting framework being achieved.Conclusion: The study suggests that use, conduct and reporting of optimal QES methods requires development, as over time the quality of reporting of QES both overall, and by specific centres, has not improved in spite of clearer reporting frameworks and important methodological developments. Further staff training in QES methods may be helpful for reviewers who are more familiar with conventional forms of systematic review if the highest standards of QES are to be achieved. There seems potential for greater use of evidence from qualitative research during guideline development. [ABSTRACT FROM AUTHOR]