Search

Objectives: To study the test-retest reliability and measurement error, construct validity, responsiveness, interpretability, and floor/ceiling effects of a Patient-Reported Outcomes Measurement Information System (PROMIS) short form designed to measure physical function in geriatric rehabilitation patients (PROMIS-PF-GR)., Design: Prospective cohort., Setting and Participants: Inpatient geriatric rehabilitation patients., Methods: We evaluated the test-retest reliability by re-administering PROMIS-PF-GR 3 to 5 days after the admission measurement. The intraclass correlation coefficient (ICC) was calculated to determine test-retest reliability; an ICC of ≥0.70 was considered sufficient. Measurement error was established by calculating the standard error of measurement and smallest detectable change. Construct validity and responsiveness were determined by testing a priori formulated hypotheses (criterion: ≥75% hypothesis not rejected). Interpretability was evaluated by calculating the minimal important change using predictive modeling and a global rating as criterion for change. Floor/ceiling effects were established by calculating the percentage patients with the minimum/maximum raw score (criterion: ≤15%) at admission and discharge., Results: A total of 207 patients participated in the study [mean ± standard deviation age (80 ± 8.3 years), 58% female]. More than one-half of patients (56%) reported to be improved during rehabilitation. The ICC was 0.79 (95% confidence interval 0.70-0.84), the standardized error of measurement was 3.8, and the smallest detectable change 10.6. None of the 4 hypotheses for construct validity were rejected; 2 out of 5 hypotheses for responsiveness were rejected. The minimal important change was 8.0 (95% confidence interval 4.1-12.5). No floor/ceiling effects were found., Conclusions and Implications: The PROMIS-PG-GR showed sufficient test-retest reliability, measurement error, and construct validity. We did not find sufficient evidence for responsiveness, which may be due to the unexplained weak correlation between the PROMIS change score and the Global Rating Scale. We still recommend the use the PROMIS-PG-GR for measuring self-reported physical function in geriatric rehabilitation., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)

Objective: PROMIS offers computerized adaptive tests (CAT) of patient-reported outcomes, using a single set of US-based IRT item parameters across populations and language-versions. The use of country-specific item parameters has local appeal, but also disadvantages. We illustrate the effects of choosing US or country-specific item parameters on PROMIS CAT T-scores., Study Design and Setting: Simulations were performed on response data from Dutch chronic pain patients (n = 1110) who completed the PROMIS Pain Behavior item bank. We compared CAT T-scores obtained with (1) US parameters; (2) Dutch item parameters; (3) US item parameters for DIF-free items and Dutch item parameters (rescaled to the US metric) for DIF items; (4) Dutch item parameters for all items (rescaled to the US metric)., Results: Without anchoring to a common metric, CAT T-scores cannot be compared. When scores were rescaled to the US metric, mean differences in CAT T-scores based on US vs. Dutch item parameters were negligible. However, 0.9%-4.3% of the T-score differences were larger than 5 points (0.5 SD)., Conclusion: The choice of item parameters can be consequential for individual patient scores. We recommend more studies of translated CATs to examine if strategies that allow for country-specific item parameters should be further investigated., (Copyright © 2021 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objective: The objective of this study was to assess the psychometric properties of the Dutch-Flemish Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function item bank in Dutch patients with chronic pain., Study Design and Setting: A bank of 121 items was administered to 1,247 Dutch patients with chronic pain. Unidimensionality was assessed by fitting a one-factor confirmatory factor analysis and evaluating resulting fit statistics. Items were calibrated with the graded response model and its fit was evaluated. Cross-cultural validity was assessed by testing items for differential item functioning (DIF) based on language (Dutch vs. English). Construct validity was evaluated by calculation correlations between scores on the Dutch-Flemish PROMIS Physical Function measure and scores on generic and disease-specific measures., Results: Results supported the Dutch-Flemish PROMIS Physical Function item bank's unidimensionality (Comparative Fit Index = 0.976, Tucker Lewis Index = 0.976) and model fit. Item thresholds targeted a wide range of physical function construct (threshold-parameters range: -4.2 to 5.6). Cross-cultural validity was good as four items only showed DIF for language and their impact on item scores was minimal. Physical Function scores were strongly associated with scores on all other measures (all correlations ≤ -0.60 as expected)., Conclusion: The Dutch-Flemish PROMIS Physical Function item bank exhibited good psychometric properties. Development of a computer adaptive test based on the large bank is warranted., (Copyright © 2017 Elsevier Inc. All rights reserved.)

Objective: There is no consensus on the best method to determine the minimal important change (MIC) of patient-reported outcomes. Recent publications recommend the use of multiple methods. Our aim was to assess whether different methods lead to consistent values for the MIC., Study Design and Setting: We used two commonly used anchor-based methods and three commonly used distribution-based methods to determine the MIC of the subscales: pain and physical functioning of the Western Ontario and McMaster University Osteoarthritis Index questionnaire in five different studies involving patients with hip or knee complaints. We repeated the anchor-based methods using relative change scores, to adjust for baseline scores., Results: We found large variation in MIC values by the same method across studies and across different methods within studies. We consider it unlikely that this variation can be explained by differences between disease groups, disease severity, or lengths of follow-up. The variation persisted when using relative change scores. It was not possible to conclude whether this variation is because of true differences in MIC values between populations or to conceptual and methodological problems of the MIC methods., Conclusion: To better disentangle these two possible explanations, the MIC methodology should be improved and standardized. In the meantime, caution is needed when interpreting and using published MIC values.

Objective: Determining "minimally important change" (MIC) facilitates the interpretation of change scores on multi-item instruments. This article focuses on how MIC values should be interpreted when applied to individual patients., Study Design and Setting: The MIC value of a hypothetical questionnaire "Q" was determined in a sample of 400 patients who improved and 100 patients who did not improve, using the receiver operating characteristic (ROC) method, and three methods to quantify the uncertainty., Results: The MIC value on questionnaire Q was 10.5. Firstly, the 95% confidence interval (CI) of the MIC value (for questionnaire Q: 5.6-14.2) quantifies the uncertainty of the estimation of the MIC value. Secondly, "how sure we are that this MIC value holds for every patient" is quantified by the values for sensitivity (74%) and specificity (91%). Thirdly, the smallest detectable change (SDC) on questionnaire Q is calculated (16.0) to consider whether the MIC value (10.5) falls outside or within the measurement error., Conclusion: For application in clinical research and practice, MIC values are always considered at the individual level, but determined in groups of patients. The interpretation comes with different forms of uncertainty. To appreciate the uncertainty, knowledge of the underlying distributions of change scores is indispensable.

Objective: To provide a practical example showing how to relate measurement error to minimal important change (MIC) to estimate sample sizes required for detecting important changes in research, and to determine the usefulness of patient-reported outcomes (PROs) in daily clinical practice., Study Design and Setting: The standard error of measurement (SEM) can be calculated as a parameter of measurement error. MIC can be determined by taking the mean change in score on the questionnaire in the subgroup of patients that reported to be "a little better" or "better" on a global rating scale minus the mean change in score in the subgroup of patients that reported to be "not changed." Based on SEM and MIC, formulas are presented that can be used to estimate sample sizes and to determine whether the questionnaire can be used to monitor changes in individual patients in daily clinical practice., Results: A fully worked-out example is presented, using Western Ontario and McMaster University Osteoarthritis Index data from a cohort of patients undergoing hip or knee replacement., Conclusion: We strongly recommend investigators to present the SEM and MIC of PROs and use these to estimate sample sizes for research and to determine the usefulness of these outcomes in daily clinical practice.