Search

Background: Th EQUATOR Network improves the quality and transparency in health research, primarily by promoting awareness and use of reporting guidelines. In 2018, the UK EQUATOR Centre launched GoodReports.org , a website that helps authors find and use reporting guidelines. This paper describes the tool's development so far. We describe user experience and behaviour of using GoodReports.org both inside and outside a journal manuscript submission process. We intend to use our findings to inform future development and testing of the tool.Methods: We conducted a survey to collect data on user experience of the GoodReports website. We cross-checked a random sample of 100 manuscripts submitted to a partner journal to describe the level of agreement between the tool's checklist recommendation and what we would have recommended. We compared the proportion of authors submitting a completed reporting checklist alongside their manuscripts between groups exposed or not exposed to the GoodReports tool. We also conducted a study comparing completeness of reporting of manuscript text before an author received a reporting guideline recommendation from GoodReports.org with the completeness of the text subsequently submitted to a partner journal.Results: Seventy percent (423/599) of survey respondents rated GoodReports 8 or more out of 10 for usefulness, and 74% (198/267) said they had made changes to their manuscript after using the website. We agreed with the GoodReports reporting guideline recommendation in 84% (72/86) of cases. Of authors who completed the guideline finder questionnaire, 14% (10/69) failed to submit a completed checklist compared to 30% (41/136) who did not use the tool. Of the 69 authors who received a GoodReports reporting guideline recommendation, 20 manuscript pairs could be reviewed before and after use of GoodReports. Five included more information in their methods section after exposure to GoodReports. On average, authors reported 57% of necessary reporting items before completing a checklist on GoodReports.org and 60% after.Conclusion: The data suggest that reporting guidance is needed early in the writing process, not at submission stage. We are developing GoodReports by adding more reporting guidelines and by creating editable article templates. We will test whether GoodReports users write more complete study reports in a randomised trial targeting researchers starting to write health research articles. [ABSTRACT FROM AUTHOR]

In composing multi-thematic questionnaires for the Lolland-Falster Health Study (LOFUS), we faced a range of challenges, for which we found limited guidance in the literature. LOFUS is a household-based population study covering multiple medical and social research areas and targeting the mixed rural-provincial population of 103,000 persons on the Danish islands Lolland and Falster. Households were randomly selected for invitation. In this paper, we describe and discuss challenges in developing the questionnaires related to stakeholders, content of the questionnaire, and the process itself. The development process was characterised by loops of learning and can be described as an iterative and incremental process. We propose recommendations to researchers and administrators involved in similar development processes, including awareness of the non-linearity and complexity of the process, a need for negotiations and navigation among multiple stakeholders, and acknowledgement of pragmatism as an inherent part of decisions made in the process. [ABSTRACT FROM AUTHOR]

The article presents a research paper which proposes a new method for evaluating reliability throughout a questionnaire by screening of systematic change in random error. Questionnaires are commonly used for medical and health care research. The paper refers to a simulation study which explores whether systematic change in reliability could be evaluated with the help of unsupervised classification of subjects by cluster analysis (CA) and estimation of intraclass correlation coefficient (ICC).

Background: Sample size calculations require effect size estimations. Sometimes, effect size estimations and standard deviation may not be readily available, particularly if efficacy is unknown because the intervention is new or developing, or the trial targets a new population. In such cases, one way to estimate the effect size is to gather expert opinion. This paper reports the use of a simple strategy to gather expert opinion to estimate a suitable effect size to use in a sample size calculation.Methods: Researchers involved in the design and analysis of clinical trials were identified at the University of Birmingham and via the MRC Hubs for Trials Methodology Research. An email invited them to participate.An online questionnaire was developed using the free online tool 'Survey Monkey©'. The questionnaire described an intervention, an electronic participant information sheet (e-PIS), which may increase recruitment rates to a trial. Respondents were asked how much they would need to see recruitment rates increased by, based on 90%. 70%, 50% and 30% baseline rates, (in a hypothetical study) before they would consider using an e-PIS in their research.Analyses comprised simple descriptive statistics.Results: The invitation to participate was sent to 122 people; 7 responded to say they were not involved in trial design and could not complete the questionnaire, 64 attempted it, 26 failed to complete it. Thirty-eight people completed the questionnaire and were included in the analysis (response rate 33%; 38/115). Of those who completed the questionnaire 44.7% (17/38) were at the academic grade of research fellow 26.3% (10/38) senior research fellow, and 28.9% (11/38) professor. Dependent upon the baseline recruitment rates presented in the questionnaire, participants wanted recruitment rate to increase from 6.9% to 28.9% before they would consider using the intervention.Conclusions: This paper has shown that in situations where effect size estimations cannot be collected from previous research, opinions from researchers and trialists can be quickly and easily collected by conducting a simple study using email recruitment and an online questionnaire. The results collected from the survey were successfully used in sample size calculations for a PhD research study protocol. [ABSTRACT FROM AUTHOR]

Background: To develop a web-based computer adaptive testing (CAT) application for efficiently collecting data regarding workers' perceptions of job satisfaction, we examined whether a 37-item Job Content Questionnaire (JCQ-37) could evaluate the job satisfaction of individual employees as a single construct. Methods: The JCQ-37 makes data collection via CAT on the internet easy, viable and fast. A Rasch rating scale model was applied to analyze data from 300 randomly selected hospital employees who participated in job satisfaction surveys in 2008 and 2009 via non-adaptive and computer-adaptive testing, respectively. Results: Of the 37 items on the questionnaire, 24 items fit the model fairly well. Person-separation reliability for the 2008 surveys was 0.88. Measures from both years and item-8 job satisfaction for groups were successfully evaluated through item-by-item analyses by using t-test. Workers aged 26 - 35 felt that job satisfaction was significantly worse in 2009 than in 2008. Conclusions: A Web-CAT developed in the present paper was shown to be more efficient than traditional computer-based or pen-and-paper assessments at collecting data regarding workers' perceptions of job content. [ABSTRACT FROM AUTHOR]

Background: The most commonly used survey methods are self-administered questionnaires, telephone interviews, and a mixture of both. But until now evidence out of randomised controlled trials as to whether patient responses differ depending on the survey mode is lacking. Therefore this study assessed whether patient responses to surveys depend on the mode of survey administration. The comparison was between mailed, self-administered questionnaires and telephone interviews. Methods: A four-armed, randomised controlled two-period change-over design. Each patient responded to the same survey twice, once in written form and once by telephone interview, separated by at least a fortnight. The study was conducted in 2003/2004 in Germany. 1087 patients taking part in the German Acupuncture Trials (GERAC cohort study), who agreed to participate in a survey after completing acupuncture treatment from an acupuncture-certified family physician for headache, were randomised. Of these, 823 (664 women) from the ages of 18 to 83 (mean 51.7) completed both parts of the study. The main outcome measure was the comparison of the scores on the 12-Item Short-Form Health Survey (SF-12) and the Graded Chronic Pain Scale (GCPS) questionnaire for the two survey modes. Results: Computer-aided telephone interviews (CATI) resulted in significantly fewer missing data (0.5%) than did mailed questionnaires (2.8%; p < 0.001). The analysis of equivalence revealed a difference between the survey modes only for the SF-12 mental scales. On average, reported mental status score was 3.5 score points (2.9 to 4.0) lower on the self-administered questionnaire compared to the telephone interview. The order of administration affected results. Patients who responded to the telephone interview first reported better mental health in the subsequent paper questionnaire (mean difference 2.8 score points) compared to those who responded to the paper questionnaire first (mean difference 4.1 score points). Conclusion: Despite the comparatively high cost of telephone interviews, they offer clear advantages over mailed self-administered questionnaires as regards completeness of data. Only items concerning mental status were dependent on the survey mode and sequence of administration. Items on physical status were not affected. Normative data for standardized telephone questionnaires could contribute to a better comparability with the results of the corresponding standardized paper questionnaires. [ABSTRACT FROM AUTHOR]

Background: A systematic review on the use of incentives to promote questionnaire return in clinical trials suggest they are effective, but not all studies have sufficient funds to use them. Promising an incentive once data are returned can reduce the cost-burden of this approach, with possible further cost-savings if the offer were restricted to reminder letters only. This study aimed to evaluate the effect of promising a monetary incentive at first mailout versus a promise on reminder letters only.Methods: This was a randomised Study Within A Trial (SWAT) nested within BUMPES, a multicentre randomised controlled trial of maternal position in the late stage of labour in women with an epidural. The follow-up questionnaire asked for information on the women's health, wellbeing and health service use one year following the birth of their baby. Women who consented to be contacted were randomised to a promise of a monetary incentive at first mailout or a promise on reminder letters only. Women were given an option of completing the questionnaire on paper or on online. The incentive was posted out on receipt of a completed questionnaire. The primary outcome was the overall return rate, and secondary outcomes were the return rate without any chasing from the study office, and the total cost of the vouchers.Results: A total of 1,029 women were randomised, 508 to the first mailout group and 518 to the reminder group. There was no evidence to suggest a difference between groups in the overall return rate (adjusted RR 1.03 (95 % CI 0.96 to 1.11), however the proportion returned without chasing was higher in the first mailout group (adjusted RR 1.22, 95 % CI 1.07 to 1.39). The total cost of the vouchers per participant was higher in the first mailout group (mean difference £4.56, 95 % CI £4.02 to £5.11).Conclusions: Offering a monetary incentive when a reminder is required could be cost-effective depending on the sample size of the study and the resources available to administer the reminder letters.Trial Registration: The BUMPES Trial is registered with Current Controlled Trials: ISRCTN35706297 , 26(th) August 2009. [ABSTRACT FROM AUTHOR]

Background: Recent developments in psychometric modeling and technology allow pooling well-validated items from existing instruments into larger item banks and their deployment through methods of computerized adaptive testing (CAT). Use of item response theory-based bifactor methods and integrative data analysis overcomes barriers in cross-instrument comparison. This paper presents the joint calibration of an item bank for researchers keen to investigate population variations in general psychological distress (GPD).Methods: Multidimensional item response theory was used on existing health survey data from the Scottish Health Education Population Survey (n = 766) to calibrate an item bank consisting of pooled items from the short common mental disorder screen (GHQ-12) and the Affectometer-2 (a measure of "general happiness"). Computer simulation was used to evaluate usefulness and efficacy of its adaptive administration.Results: A bifactor model capturing variation across a continuum of population distress (while controlling for artefacts due to item wording) was supported. The numbers of items for different required reliabilities in adaptive administration demonstrated promising efficacy of the proposed item bank.Conclusions: Psychometric modeling of the common dimension captured by more than one instrument offers the potential of adaptive testing for GPD using individually sequenced combinations of existing survey items. The potential for linking other item sets with alternative candidate measures of positive mental health is discussed since an optimal item bank may require even more items than these. [ABSTRACT FROM AUTHOR]

Background: There are inherent methodological challenges in the measurement of mental health problems in longitudinal research. There is constant development in definitions, taxonomies and demands concerning the properties of mental health measurements. The aim of this paper was to construct composite measures of mental health problems (according to today's standard) from single questionnaire items devised in the early 1980s, and to evaluate their internal consistency and factorial invariance across the life course using the Northern Swedish Cohort.Methods: All pupils in the last year of compulsory school in Luleå in 1981 (n = 1083) form a prospective cohort study where the participants have been followed with questionnaires from the age of 16 (in 1981) until the age of 43 (in 2008). We created and tested the following composite measures from self-reports at each follow-up: depressive symptoms, anxiety symptoms, functional somatic symptoms, modified GHQ and positive health. Validity and internal consistency were tested by confirmatory factor analysis, including tests of factorial invariance over time.Results: As an overall assessment, the results showed that the composite measures (based on more than 30-year-old single item questions) are likely to have acceptable factorial invariance as well as internal consistency over time.Conclusions: Testing the properties of the mental health measures used in older studies according to the standards of today is of great importance in longitudinal research. Our study demonstrates that composite measures of mental health problems can be constructed from single items which are more than 30 years old and that these measures seem to have the same factorial structure and internal consistency across a significant part of the life course. Thus, it can be possible to overcome some specific inherent methodological challenges in using historical data in longitudinal research. [ABSTRACT FROM AUTHOR]

Background: Using patient-reported outcome measures (PROMs) to assess Quality of Life (QoL) is well established, but commonly-used PROM item-sets do not necessarily capture what all respondents consider important. Measuring complex constructs is particularly difficult in randomised controlled trials (RCTs). The Mother-Generated Index (MGI) is a validated antenatal and postnatal QoL instrument in which the variables and scores are completely respondent-driven. This paper reports on the feasibility and acceptability of the MGI in an RCT, and compares the resulting variables and QoL scores with more commonly used instruments.Methods: The single-page MGI was included at the end of a ten page questionnaire pack and posted to the RCT participants at baseline (28-32 weeks' gestation) and follow-up (six weeks postnatal). Feasibility and acceptability were assessed by ease of administration, data entry and completion rates. Variables cited by women were analysed thematically. MGI QoL scores were compared with outcomes from the EQ-5D-3 L; Edinburgh Postnatal Depression Scale; Satisfaction With Life Scale; and State Trait Anxiety Inventory.Results: Six hundred and seventy eight pregnant women returned the pack at baseline; 668 completed the MGI (98.5%); 383/400 returns at follow up included a completed MGI (95.7%). Quantitative data were scanned into SPSS using a standard data scanning system, and were largely error-free; qualitative data were entered manually. The variables recorded by participants on the MGI forms incorporated many of those in the comparison instruments, and other outcomes commonly used in intrapartum trials, but they also revealed a wider range of issues affecting their quality of life. These included financial and work-related worries; moving house; and concerns over family illness and pets. The MGI scores demonstrated low-to-moderate correlation with other tools (all r values p < .01).Conclusions: Without face-to-face explanation and at the end of a long questionnaire, the MGI was feasible to use, and acceptable to RCT participants. It allowed individual participants to include issues that were important to them, but which are not well captured by existing tools. The MGI unites the explanatory power of qualitative research with the comparative power of quantitative designs, is inexpensive to administer, and requires minimal linguistic and conceptual translation.Trial Registration: ISRCTN27575146 (date assigned 23 March 2011). [ABSTRACT FROM AUTHOR]

Background: An increasing number of research designs are using text messaging (SMS) as a means of self-reported symptom and outcome monitoring in a variety of long-term health conditions, including severity ratings of depressed mood. The validity of such a single item SMS score to measure latent depression is not currently known and is vital if SMS data are to inform clinical evaluation in the future. Methods: A sub-set of depressed participants in the UK ACUDep trial submitted a single SMS text score (R-SMS-DS) between 1 and 9 on how depressed they felt around the same time as completing the PHQ-9 depression questionnaire on paper at 3 months follow-up of the trial. Exploratory categorical data factor analysis (EFA) was used to ascertain the alignment of R-SMS-DS scores with the factor structure of the PHQ-9. Any response bias with regard to age or gender was assessed by differential item functioning (DIF) analysis. Results: Depression scores based on the PHQ-9 and R-SMS-DS at 3 months were available for 337 participants (74 % female; mean age: 42 years, SD=11.1), 213 of which completed the two outcomes within 6 days of each other. R-SMS-DS scores aligned with the underlying latent depression of the PHQ-9 (factor loading of 0.656) and in particular its affective rather than somatic dimension. The R-SMS-DS score was most strongly correlated with depressed mood (r = 0.607), feeling bad about oneself (r = 0.588) and anhedonia (r = 0.573). R-SMS-DS responses were invariant with respect to gender (p = 0.302). However, there was some evidence for age related response bias (p = 0.031), with older participants being more likely to endorse lower R-SMS-DS scores than younger ones. Conclusions: The R-SMS-DS used in the ACUDep trial was found to be a valid measure of latent affective depression with no gender related response bias. This text message item may therefore represent a useful assessment and monitoring tool meriting evaluation in further research. For future study designs we recommend the collection of outcome data by new health technologies in combination with gold standard instruments to ensure concurrent validity. [ABSTRACT FROM AUTHOR]

Background Self-administered questionnaires or interviews and medical records are often used as sources of research data; thus it is essential to evaluate their concordance and reliability. The aim of this paper was to assess the concordance between medical and behavioral data obtained from medical records and interview questionnaires in two correctional facilities. Methods Medical record and interview data were compared for 679 inmates from one male and one female maximum security prison between April 2010 and February 2013. Gender non-stratified and gender-stratified analyses were conducted in SPSS to calculate the prevalence and kappa coefficient scores (κ) for medical (e.g., HIV, diabetes, hypertension) and behavioral (e.g., smoking, drug use, tattoos) conditions. Sensitivity/specificity between medical records and interview were calculated in the gender non-stratified data. Results In the gender non-stratified analysis, κ score for HIV, hepatitis C, diabetes, asthma, and history of tattoos had strong or good concordance (0.66-0.89). Hypertension, renal/kidney disease, cigarette smoking, antibiotic use in the last 6 months, and cocaine use ever were moderately correlated (0.49-0.57). Both history of any illicit drug use ever (0.36) and marijuana use ever (0.23) had poor concordance. Females had higher κ scores and prevalence rates than males overall. Medical conditions were reported more frequently in medical records and behavioral conditions had higher prevalence in interviews. Sensitivity for medical conditions in the combined facility data ranged from 50.0% to 86.0% and 48.2% to 85.3% for behavioral conditions whereas specificity ranged from 95.9% to 99.5% for medical conditions and 75.9% to 92.8% for behavioral conditions. Conclusion Levels of agreement between medical records and self-reports varied by type of factor. Medical conditions were more frequently reported by chart review and behavioral factors more frequently by self-report. Data source used may need to be chosen carefully depending upon the type of information sought. [ABSTRACT FROM AUTHOR]

The article presents a research paper which focuses on a methodological approach used in the pre-validation development of new instruments for measuring outcomes in randomized controlled trials. It discusses the importance of measurement instruments like item generation, item reduction and question formatting. The new method uses the example of the Aberdeen Glaucoma Questionnaire (AGQ) and consists of five steps including generation of a pool of items; item de-duplication and item reduction.

Background: There is an increasing trend towards lower participation in questionnaire surveys. This reduces representativeness, increases costs and introduces particular challenges to longitudinal surveys, as researchers have to use complex statistical techniques which attempt to address attrition. This paper describes a trial of incentives to retain longitudinal survey cohorts from ages 16 to 20, to question them on the sensitive topic of sexual health. Methods: A longitudinal survey was conducted with 8,430 eligible pupils from two sequential year groups from 25 Scottish schools. Wave 1 (14 years) and Wave 2 (16 years) were conducted largely within schools. For Wave 3 (18 years), when everyone had left school, the sample was split into 4 groups that were balanced across predictors of survey participation: 1) no incentive; 2) chance of winning one of twenty-five vouchers worth £20; 3) chance of winning one £500 voucher; 4) a definite reward of a £10 voucher sent on receipt of their completed questionnaire. Outcomes were participation at Wave 3 and two years later at Wave 4. Analysis used logistic regression and adjusted for clustering at school level. Results: The only condition that had a significant and beneficial impact for pupils was to offer a definite reward for participation (Group 4). Forty-one percent of Group 4 participated in Wave 3 versus 27% or less for Groups 1 to 3. At Wave 4, 35% of Group 4 took part versus 25% or less for the other groups. Similarly, 22% of Group 4 participated in all four Waves of the longitudinal study, whereas for the other three groups it was 16% or less that participated in full. Conclusions: The best strategy for retaining all groups of pupils and one that improved retention at both age 18 and age 20 was to offer a definite reward for participation. This is expensive, however, given the many benefits of retaining a longitudinal sample, we recommend inclusion of this as a research cost for cohort and other repeatcontact studies. [ABSTRACT FROM AUTHOR]

Background: Research questionnaires are not always translated appropriately before they are used in new temporal, cultural or linguistic settings. The results based on such instruments may therefore not accurately reflect what they are supposed to measure. This paper aims to illustrate the process and required steps involved in the cross-cultural adaptation of a research instrument using the adaptation process of an attitudinal instrument as an example. Methods: A questionnaire was needed for the implementation of a study in Norway 2007. There was no appropriate instruments available in Norwegian, thus an Australian-English instrument was cross-culturally adapted. Results: The adaptation process included investigation of conceptual and item equivalence. Two forward and two back-translations were synthesized and compared by an expert committee. Thereafter the instrument was pretested and adjusted accordingly. The final questionnaire was administered to opioid maintenance treatment staff (n=140) and harm reduction staff (n=180). The overall response rate was 84%. The original instrument failed confirmatory analysis. Instead a new two-factor scale was identified and found valid in the new setting. Conclusions: The failure of the original scale highlights the importance of adapting instruments to current research settings. It also emphasizes the importance of ensuring that concepts within an instrument are equal between the original and target language, time and context. If the described stages in the cross-cultural adaptation process had been omitted, the findings would have been misleading, even if presented with apparent precision. Thus, it is important to consider possible barriers when making a direct comparison between different nations, cultures and times. [ABSTRACT FROM AUTHOR]

Background: Data for health surveys are often collected using either mailed questionnaires, telephone interviews or a combination. Mode of data collection can affect the propensity to refuse to respond and result in different patterns of responses. The objective of this paper is to examine and quantify effects of mode of data collection in health surveys. Methods: A stratified sample of 4,000 adults residing in Denmark was randomised to mailed questionnaires or computer-assisted telephone interviews. 45 health-related items were analyzed; four concerning behaviour and 41 concerning self assessment. Odds ratios for more positive answers and more frequent use of extreme response categories (both positive and negative) among telephone respondents compared to questionnaire respondents were estimated. Tests were Bonferroni corrected. Results: For the four health behaviour items there were no significant differences in the response patterns. For 32 of the 41 health self assessment items the response pattern was statistically significantly different and extreme response categories were used more frequently among telephone respondents (Median estimated odds ratio: 1.67). For a majority of these mode sensitive items (26/32), a more positive reporting was observed among telephone respondents (Median estimated odds ratio: 1.73). The overall response rate was similar among persons randomly assigned to questionnaires (58.1%) and to telephone interviews (56.2%). A differential nonresponse bias for age and gender was observed. The rate of missing responses was higher for questionnaires (0.73 - 6.00%) than for telephone interviews (0 - 0.51%). The "don't know" option was used more often by mail respondents (10 - 24%) than by telephone respondents (2 - 4%). Conclusion: The mode of data collection affects the reporting of self assessed health items substantially. In epidemiological studies, the method effect may be as large as the effects under investigation. Caution is needed when comparing prevalences across surveys or when studying time trends. [ABSTRACT FROM AUTHOR]

Background: Missing data can introduce bias in the results of randomised controlled trials (RCTs), but are typically unavoidable in pragmatic clinical research, especially when patient reported outcome measures (PROMs) are used. Traditionally applied to the composite PROMs score of multi-item instruments, some recent research suggests that multiple imputation (MI) at the item level may be preferable under certain scenarios. This paper presents practical guidance on the choice of MI models for handling missing PROMs data based on the characteristics of the trial dataset. The comparative performance of complete cases analysis, which is commonly used in the analysis of RCTs, is also considered.Methods: Realistic missing at random data were simulated using follow-up data from an RCT considering three different PROMs (Oxford Knee Score (OKS), EuroQoL 5 Dimensions 3 Levels (EQ-5D-3L), 12-item Short Form Survey (SF-12)). Data were multiply imputed at the item (using ordinal logit and predicted mean matching models), sub-scale and score level; unadjusted mean outcomes, as well as treatment effects from linear regression models were obtained for 1000 simulations. Performance was assessed by root mean square errors (RMSE) and mean absolute errors (MAE).Results: Convergence problems were observed for MI at the item level. Performance generally improved with increasing sample sizes and lower percentages of missing data. Imputation at the score and subscale level outperformed imputation at the item level in small sample sizes (n ≤ 200). Imputation at the item level is more accurate for high proportions of item-nonresponse. All methods provided similar results for large sample sizes (≥500) in this particular case study.Conclusions: Many factors, including the prevalence of missing data in the study, sample size, the number of items within the PROM and numbers of levels within the individual items, and planned analyses need consideration when choosing an imputation model for missing PROMs data. [ABSTRACT FROM AUTHOR]