Search

Objectives There is growing interest in condition-specific preference measures, including the European Organisation for Research and Treatment of Cancer Quality of Life Utility Measure-Core 10 Dimensions (QLU-C10D). This research assessed the implications of using utility indices on the basis of the EQ-5D-3L, a mapping of EQ-5D-3L to the EQ-5D-5L, and the QLU-C10D, and compared their psychometric properties. Methods Data were taken from 8 phase 3 randomized controlled trials of nivolumab with or without ipilimumab for the treatment of solid tumors. Utilities for progression-related states were calculated using the UK and English value sets and incremental quality-adjusted life-years (QALYs) derived from established UK cost-effectiveness models. The psychometric properties of the utility indices were assessed using pooled trial data. Results Compared with the EQ-5D-3L index, the mapped EQ-5D-5L index yielded an average of 6% more and the QLU-C10D index an average of 2% fewer incremental QALYs for nivolumab versus comparators. All indices could differentiate between groups defined by performance status, cancer stage, or self-reported health status at baseline and detect meaningful changes in performance status, tumor response, health status, and quality of life over approximately 12 weeks of treatment. Conclusions The lower QALY yield of the QLU-C10D was balanced by evidence of greater validity and responsiveness. Benefits gained from using the QLU-C10D may be apparent when treatments affect targeted symptoms and functional aspects, including sleep, bowel function, appetite, nausea, and fatigue. The observed differences in QALYs may not be sufficiently large to affect health technology assessment decisions.

Objectives The original 3-level EQ-5D (EQ-5D-3L) includes 5 dimensions with 3 levels of problems per dimension. Since 2010, a more sensitive version with 5 levels of problems per dimension (EQ-5D-5L) has become available. Population value sets have been developed for both versions of the questionnaire. The objective of this research was to develop a mapping function to link EQ-5D-3L responses to value sets for the EQ-5D-5L. Methods Various algorithms were developed to link EQ-5D-3L and EQ-5D-5L responses using data from an observational study including members of 10 subgroups (N = 3580) who completed both versions of the questionnaire. Nonparametric and ordinal logistic regression models were fit to the data and compared using Akaike’s information criterion (AIC) as well as the mean absolute error and root mean squared error of predictions. Results were contrasted qualitatively and quantitatively with those of an alternative copula-based approach. Results Including indicants of problems for other EQ-5D-3L dimensions as regressors in the modeling yielded the greatest improvement in prediction accuracy. Adding age and gender lowered the AIC without improving predictions, while including a latent factor lowered the AIC further and slightly improved predictive accuracy. Models that conditioned on problems in other EQ-5D-3L dimensions yielded more accurate predictions than the alternative copula-based approach in subgroups defined by age and gender. Conclusion We present novel algorithms to map EQ-5D-3L responses to EQ-5D-5L value sets. The recommended approach is based on an ordinal logistic regression that disregards age and gender and accounts for unobserved heterogeneity using a latent factor.

Aim The EQ-5D is a generic measure of health that is widely applied for health economic and non-economic purposes. Population norms can be used to facilitate the interpretation of EQ-5D data. The objective of this study was to develop a set of pooled normative EQ-5D-3L values for the five largest European economies (EUR5). Methods EQ-5D-3L index values based on the time trade-off (TTO) were available for all EUR5 countries (n = 21,425): France, Germany, Italy, Spain, and the United Kingdom (UK). Country-specific data sets were aggregated and weighted to facilitate the derivation of norms for gender and age groups. Analyses included equal weighting and weighting by population and economy size. Norms were also calculated using the European visual analog scale-based value set (European VAS), the EQ VAS and separately by dimension. Results Pooled mean (SD) population weighted TTO values for males/females were 0.967 (0.122)/0.959 (0.118) for ages 18–24; 0.965 (0.096)/0.954 (0.117) for ages 25–34; 0.943 (0.165)/0.936 (0.169) for ages 35–44; 0.934 (0.150)/0.921 (0.157) for ages 45–54; 0.896 (0.188)/0.875 (0.197) for ages 55–64; 0.900 (0.158)/0.839 (0.218) for ages 65–74; and 0.830 (0.234)/0.756 (0.291) for ages 75 and older. Mean values decreased and variance increased with age; females had slightly lower mean values than males across all age bands. The unequal weighting approaches produced similar point estimates with smaller variances. Mean values for the European VAS were slightly lower than those for the TTO-based index. Discussion Normative EQ-5D-3L values can be used to benchmark the outcomes of treated patients against the health of the general population. EUR5 norms may be useful in research applications inferring to Europe or the European Union as a whole, particularly when sample size precludes analysis at the country level.

Introduction Discrete choice experiments (DCE) are increasingly being conducted using online panels. However, the comparability of such DCE-based preferences to traditional modes of data collection (e.g., in-person) is not well established. In this study, supervised, face-to-face DCE was compared with its unsupervised, online facsimile on face validity, respondent behavior, and modeled preferences. Methods Data from face-to-face and online EQ-5D-5L health state valuation studies were compared, in which each used the same experimental design and quota sampling procedure. Respondents completed 7 binary DCE tasks comparing 2 EQ-5D-5L health states presented side by side (health states A and B). Data face validity was assessed by comparing preference patterns as a function of the severity difference between 2 health states within a task. The prevalence of potentially suspicious choice patterns (i.e., all As, all Bs, and alternating As/Bs) was compared between studies. Preference data were modeled using multinomial logit regression and compared based on dimensional contribution to overall scale and importance ranking of dimension-levels. Results One thousand five Online respondents and 1,099 face-to-face screened (F2FS) respondents were included in the main comparison of DCE tasks. Online respondents reported more problems on all EQ-5D dimensions except for Mobility. The face validity of the data was similar between comparators. Online respondents had a greater prevalence of potentially suspicious DCE choice patterns ([Online]: 5.3% [F2FS] 2.9%, P = 0.005). When modeled, the relative contribution of each EQ-5D dimension differed between modes of administration. Online respondents weighed Mobility more importantly and Anxiety/Depression less importantly. Discussion Although assessments of face validity were similar between Online and F2FS, modeled preferences differed. Future analyses are needed to clarify whether differences are attributable to preference or data quality variation between modes of data collection.

Web-based surveys are increasingly utilized for health valuation studies but may be more prone to lack of engagement and, therefore, poor data validity. The objective of this study was to evaluate the effect of imposed engagement (i.e., at least three trade-offs) in the composite time trade-off (cTTO) task. The EQ-5D-5L valuation study protocol and study design were adapted for online, unsupervised completion in two arms: base case and engagement. Validity of preferences was assessed using the prevalence of inconsistent valuations and expected patterns of TTO values. Respondent task engagement was measured using time per task. Value sets were generated using linear regression with a random intercept (RILR). The base case (n = 501) and engagement arms (n = 504) clustered at different TTO values: [base case] 0, 1; [engagement] -0.5, 0.45, 0.6. Mean TTO values were lower for the engagement arm. Engagement respondents did not spend more time per TTO task: [base case] 63.3 s (SD 77.9 s); [engagement] 64.7 s (SD 73.3 s); p = 0.36. No significant difference was found between arms for prevalence of respondents with at least one inconsistent TTO value: [base case] 61.1%; [engagement] 63.5%; p = 0.43. Both value sets had significant intercepts far from 1: [base case] 0.846; [engagement] 0.783. The relative importance of the EQ-5D dimensions also differed between arms. Both online arms had poor quality data. A minimum trade-off threshold did not improve engagement nor face validity of the data, indicating that modifications to the number of iterations are insufficient alone to improve data quality/validity of online TTO studies.

Objectives In the CheckMate 141 trial (NCT02105636), nivolumab demonstrated survival, health-related quality of life, and healthcare resource utilization benefits vs single-agent therapy of investigator’s choice (IC) (methotrexate, docetaxel or cetuximab) in patients with platinum-refractory recurrent/metastatic squamous cell carcinoma of the head and neck (R/M SCCHN). We assessed between-treatment differences in quality-adjusted time without symptoms of disease progression or toxicity (Q-TWiST). Methods Survival data from CheckMate 141 (nivolumab, n = 240; IC, n = 121) was partitioned into toxicity (TOX), time without symptoms or toxicity (TWiST), and relapse (REL). TOX was defined as time spent with all-cause grade 3–4 adverse events after randomization, before disease progression. TWiST was defined as time not in TOX or REL. REL was defined as time between disease progression and death. Utility values derived from three-level EuroQol five-dimensional questionnaire data from CheckMate 141 were used to calculate Q-TWiST as the utility-weighted sum of the mean duration in each health state. Results The between-group difference in Q-TWiST score was 1.23 months (95% confidence interval 1.17–1.29) favoring nivolumab (p 10%) in quality-adjusted survival vs standard of care in patients with R/M SCCHN.

Objectives The introduction of innovative, high-cost oncology treatments, coupled with mounting budgetary pressures, necessitates value trade-offs across cancer types. Defining value is critical to informing decision-making. A cost-value analysis tool was used to assess relative clinical value from a US perspective using multiple outcome metrics for a variety of metastatic cancers. Methods Literature published (January 1, 2000-August 31, 2016) was reviewed to identify outcome metrics for approved treatments for metastatic cancers. Data were extracted or derived for median and mean overall survival (OS), landmark survival rates, and other survival metrics, and compared across treatments vs their respective trial comparators, with and without considering costs. Results Reported survival metrics varied by agent within cancer type. For treatment of prostate cancer, abiraterone yielded the highest improvement in 1-year survival rate (13.7%, previously treated), whereas enzalutamide yielded the highest median OS improvement (4.8 months, previously treated) and sipuleucel-T, the highest mean OS improvement (3.6 months, previously untreated) vs their respective trial comparators. For treatment of non-small cell lung cancer vs their respective trial comparators, nivolumab yielded the highest improvement in mean OS (11.9 months) and 3-year survival rate (12.6%), each in previously treated squamous disease, whereas afatinib yielded the highest median OS improvement (4.1 months, previously untreated EGFR del19 and L858R mutants). Cost-value analysis results varied with the applied survival metric. Conclusions Although median OS is the traditional gold standard oncology efficacy metric, it fails to capture long-term survival benefits-the ultimate goal of cancer treatment-offered by new treatment modalities. Diverse metrics are needed for comprehensive value assessments of cancer therapies.

Objective To derive a US-based value set for the EQ-5D-5L questionnaire using an international, standardized protocol developed by the EuroQol Group. Methods Respondents from the US adult population were quota-sampled on the basis of age, sex, ethnicity, and race. Trained interviewers guided participants in completing composite time trade-off (cTTO) and discrete choice experiment (DCE) tasks using the EuroQol Valuation Technology software and routine quality control measures. Data were modeled using a Tobit model for cTTO data, a mixed logit model for DCE data, and a hybrid model that combined cTTO and DCE data. Model performance was compared on the basis of logical ordering of coefficients, statistical significance, parsimony, and theoretical considerations. Results Of 1134 respondents, 1062, 1099, and 1102 respondents provided useable cTTO, DCE, and cTTO or DCE responses, respectively, on the basis of quality control criteria and interviewer judgment. Respondent demographic characteristics and health status were similar to the 2015 US Census. The Tobit model was selected as the preferred model to generate the value set. Values ranged from −0.573 (55 555) to 1 (11 111), with 20% of all predicted health states scores less than 0 (ie, worse than dead). Conclusions A societal value set for the EQ-5D-5L was developed that can be used for economic evaluations and decision making in US health systems. The internationally established, standardized protocol used to develop this US-based value set was recommended by the EuroQol Group and can facilitate cross-country comparisons.

Background The EORTC QLU-C10D is a multiattribute utility measure derived from the cancer-specific quality-of-life questionnaire, the EORTC QLQ-C30. The QLU-C10D contains 10 dimensions (physical, role, social and emotional functioning, pain, fatigue, sleep, appetite, nausea, bowel problems). The objective of this study was to develop a United States value set for the QLU-C10D. Methods A US online panel was quota recruited to achieve a representative sample for sex, age (≥18 y), race, and ethnicity. Respondents undertook a discrete choice experiment, each completing 16 choice-pairs, randomly assigned from a total of 960 choice-pairs. Each pair included 2 QLU-C10D health states and duration. Data were analyzed using conditional logistic regression, parameterized to fit the quality-adjusted life-year framework. Utility weights were calculated as the ratio of each dimension-level coefficient to the coefficient for life expectancy. Results A total of 2480 panel members opted in, 2333 (94%) completed at least 1 choice-pair, and 2273 (92%) completed all choice-pairs. Within dimensions, weights were generally monotonic. Physical functioning, role functioning, and pain were associated with the largest utility weights. Cancer-specific dimensions, such as nausea and bowel problems, were associated with moderate utility decrements, as were general issues such as problems with emotional functioning and social functioning. Sleep problems and fatigue were associated with smaller utility decrements. The value of the worst health state was 0.032, which was slightly greater than 0 (equivalent to being dead). Conclusions This study provides the US-specific value set for the QLU-C10D. These estimated health state scores, based on responses to the EORTC QLQ-C30 questionnaire, can be used to evaluate the cost-utility of oncology treatments.

It is important that patient-reported outcome (PRO) measures used to assess cancer therapies adequately capture the benefits and risks experienced by patients, particularly when adverse event profiles differ across therapies. This study explores the case for augmenting preference-based utility measures to capture the impact of cancer treatment-related symptoms. Additional cancer treatment-related items could be specific (e.g., rash) or global. While specific items are easier to describe and understand, their use may miss rarer symptoms and those that are currently unknown but will arise from future medical advancements. The appropriate number of additional items, the independence of those items, and their impact on the psychometric properties of the core instrument require consideration. Alternatively, a global item could encompass all potential treatment-related symptoms, of any treatments for any disease. However, such an item may not be well understood by general public respondents in valuation exercises. Further challenges include the decision about whether to generate de novo value sets for the modified instrument or to map to existing tariffs. The fluctuating and transient nature of treatment-related symptoms may be inconsistent with the methods used in conventional valuation exercises. Fluctuating symptoms could be missed by sub-optimal measure administration timing. The addition of items also poses double-counting risks. In summary, the addition of treatment-related symptom items could increase the sensitivity of existing utility measures to capture known and unknown treatment effects in oncology, while retaining the core domains. However, more research is needed to investigate the challenges, particularly regarding valuation.

Aim: To assess the cost-effectiveness of nivolumab monotherapy for recurrent/metastatic (R/M) squamous cell carcinoma of the head and neck (SCCHN) in the US. Methods: We constructed a cohort-based partitioned survival model for three health states (progression-free, progressed disease, and death). Using overall survival and progression-free survival data from the nivolumab and investigator’s choice (IC) arms of the CheckMate 141 study, the proportion of patients in each health state was estimated by parametric modeling over a 25-year period. Cost, utility, adverse event, and disease management data inputs were obtained from relevant literature and applied to patients in each health state. A scenario analysis was conducted assuming increased uptake of subsequent immunotherapies. A one-way deterministic sensitivity analysis assessed the impact of variation in multiple parameters. A probabilistic sensitivity analysis in which probabilistic distributions were applied to each input during 1,000 model iterations was also conducted. Results: Total costs incurred were higher with nivolumab ($101,552) than with IC ($38,067). Nivolumab was associated with a higher number of life-years (LY; 1.21) and quality-adjusted life-years (QALYs; 0.89), compared with IC (0.68 and 0.42, respectively). The incremental cost-effectiveness ratio for nivolumab compared with IC was $134,438 per QALY, and this remained qualitatively similar when increased uptake of subsequent immunotherapies was assumed ($129,603 per QALY). Sensitivity analyses supported these findings. Conclusions: These results suggest that, at a willingness-to-pay threshold of $150,000 per QALY, nivolumab is a cost-effective option for therapy of SCCHN in the US.

Aims: In 2016, nivolumab and pembrolizumab were approved for the treatment of squamous cell carcinoma of the head and neck (SCCHN) following progression after initial platinum-based therapy. We sou...

Objective The D1 model that was developed to predict US societal preferences for EQ-5D health states addressed several important conceptual and statistical issues. However, it has been criticized for being too complex, failing to account for the nonnormal distribution of health state values, and the transformation of preferences for worse-than-death health states before estimation. This research was conducted to develop an improved model for predicting median preferences for EQ-5D health states for the US population. Methods Probability-weighted least absolute deviations regression was used to fit models to the time trade-off data collected in the US Valuation of the EQ-5D Health States study. No transformation was applied to the values for states considered worse than death. Several model specifications that differed with respect to explanatory variables were evaluated using two-sample cross-validation. Results The best-fitting model included only fixed effects for moderate or severe problems in each of the 5 EQ-5D dimensions and excluded a constant. This specification yielded rank correlations between observed and predicted values and median observed and predicted values of 0.635 and 0.991, respectively, as well as a median absolute error of 0.026. The predicted median preferences ranged from 1.00 for full health, to –0.81 for the worst possible health state. Conclusions Due to its simplicity and robustness, a median model is superior to other models for predicting US population preferences for EQ-5D health states. The predictions of this model are suggested for use in applications that require US societal health state values.

The authors recently introduced a new preference-based scoring function for the EQ-5D (D1 model) based on time tradeoff valuations from the general adult US population. In this study, they compared the EQ-5D index scores derived from the US (D1) algorithm to the more familiar UK (N3) algorithm. They compared preference-based EQ-5D index scores for all possible EQ-5D health states and differences in EQ-5D index scores between pairs of EQ-5D health states predicted by the D1 and N3 models. The responsiveness of D1- and N3-predicted EQ-5D index scores was assessed using simulated transitions between EQ-5D health states. The mean (SD) EQ-5D index scores for all 243 health states predicted by the D1 and N3 models were 0.37 (0.23) and 0.14 (0.31), respectively. The mean (SD) absolute difference in EQ-5D index scores for all 29,403 pairs of health states was 0.25 (0.19) and 0.35 (0.27), according to the D1 and N3 models, respectively. The D1 and N3 models were consistent in predicting gains/losses for 27,592 (94%) transitions between EQ-5D health state pairs; Cohen effect size, calculated using these 27,592 consistent transitions, was 1.58 and 1.59 for the D1 and N3 models, respectively. Based on these simulation results, it appears that the D1 model would lead to smaller gains in quality-adjusted life years than the N3 model; however, their responsiveness appears to be similar. Empirical studies are needed to examine whether these 2 EQ-5D scoring functions would lead to different conclusions in cost-utility analyses.

≥Ying Zhang,1 Trong Kim Le,1 James W Shaw,2 Srividya Kotapati31Center for Observational Research and Data Sciences, Worldwide Health Economics and Outcomes Research, Bristol-Myers Squibb Research and Development, Hopewell, NJ, USA; 2Worldwide Health Economics and Outcomes Research, Bristol-Myers Squibb Research and Development, Princeton, NJ, USA; 3Worldwide Health Economics and Outcomes Research, Bristol-Myers Squibb Research and Development, Wallingford Center, CT, USABackground: This study examines real-world drug utilization patterns, health care resource use, and costs among patients receiving adjuvant treatment with IFN versus patients receiving no treatment ("observation") for malignant melanoma following surgery.Methods: A retrospective cohort study was conducted using administrative claims from Truven Health Analytics (MarketScan®) to identify all adjuvant melanoma patients (aged ≥18 years) diagnosed between June 2007 and June 2011 who had a lymph node dissection (ie, index surgery) and were treated with IFN or subsequently observed. Health care resource use and costs of services were converted to 2012 US dollars and were evaluated and compared using multivariable regression.Results: Of 1,999 eligible subjects with melanoma surgery claims, 179 (9.0%) were treated with IFN and 1,820 (91.0%) were observed. The median duration (days) and number of doses of IFN therapy were 73 and 36, respectively. Among IFN-treated patients, only 10.6% completed ≥80% of maintenance therapy. The total average cost for patients treated with IFN was US$60,755±$3,972 (n=179); significantly higher than for patients undergoing observation ($31,641±$2,471; P

OBJECTIVE: To evaluate the effect of a closed formulary on pharmaceutical utilization and expenditures in a noncontinuously eligible population. DESIGN: A pretest-posttest design was used to compare changes in pharmaceutical utilization and expenditures between the formulary and a nonformulary group in the nine months before formulary implementation and the nine months after. SETTING: Two government employer plans in the eastern United States. PARTICIPANTS: Noncontinuously eligible individual plan members were included in the study if they had been enrolled in their plan for at least one month in both the pre- and the post periods. MAIN OUTCOME MEASURES: Mean pharmaceutical utilization and expenditures per member per month (PM PM). RESULTS: Formulary cases (n=1,542) were matched on age and baseline pharmaceutical utilization with controls who were not in a formulary group plan. These cases had a higher generic fill rate, lower mean total claims PMPM, and lower mean brand claims PMPM in the post-period, co...

• Patient-reported outcome (PRO) instruments used to assess QoL among patients with recurrent GBM included the European Organisation for Research and Treatment of Cancer (EORTC) Core Quality of Life Questionnaire (QLQ-C30), the EORTC QLQ Brain Cancer Module (EORTC QLQBN20), the Functional Assessment of Cancer Therapy (FACT)–General (FACT-G), the FACT–Brain (FACT-Br), and the 36-Item Short-Form Health Survey (SF-36) (Table 2)

The symptoms of onychomycosis and their impact on personal appearance are important determinants of patients' perceptions of their own health. The effect of onychomycosis is greater on psychosocial than physical functioning and is directly related to the extent of nail involvement. This review identified a number of disease-targeted questionnaires that have been developed to assess the impact of onychomycosis and its treatment on health-related quality of life (HR-QOL). The instruments differ considerably in the extent to which they have been psychometrically tested. Most have been shown to provide both valid and reliable measurement of HR-QOL in patients with onychomycosis of the toes. However, not all have been tested in patients with fingernail disease. The generic and disease-targeted scales of most of the available questionnaires exhibit poor variability, which may limit their responsiveness to clinically important change. Only one instrument, the Onychomycosis Disease-Specific Questionnaire, has been incorporated into a clinical trial. Thus, information relating to the impact of onychomycosis treatment on HR-QOL is limited. Future efforts should focus on the systematic evaluation of existing questionnaires in all populations for which they were developed. In addition, it is important that disease-targeted measures be included in future clinical trials so that a better understanding of the impact of onychomycosis treatment on patient HR-QOL may be gained.

BACKGROUND: Migraine headache affects approximately 9 million Americans. It is a chronic disease and is often associated with a high rate of disability. OBJECTIVE: The purpose of this research was to develop a model for predicting the incident number of migraine cases in the Idaho Medicaid population over a 5-year period. METHODS: Incident migraine cases between January 1994 and December 1998 were identified from the Idaho Medicaid claims database using migraine-specific ICD-9-CM codes or migraine-specific pharmacy claims. Data were collected on the total number of new migraine cases according to age, race, gender, and year. A negative binomial model was developed to predict incident migraine cases from the other variables. All model independents were treated as categorical variables. Two hundred sixty-six Medicaid recipients between the ages of 5 and 104 were included in the analysis. RESULTS: A test for overdispersion revealed that use of a negative binomial model was appropriate. Age, race, gender, and year were all determined to be significant effects by likelihood-ratio test. The greatest incidence of migraine was observed among persons 30–34 years of age. Controlling for other effects, the incidence of migraine fell with increasing age> 34 or decreasing age

Background The European Organization for the Research and Treatment of Cancer Quality of Life Questionnaire (QLQ-C30) is a widely used quality-of-life measure in oncology. The ability to translate QLQ-C30 responses into utility scores would further expand its use in medical decision-making. The aims of this study were to: 1) map QLQ-C30 responses onto patient time trade-off utility scores; and 2) compare a multiattribute approach to a global evaluation approach to modeling utility scores. Methods Two distinct approaches were applied to data from 1432 cancer patients. The multiattribute approach used psychometric analysis and expert input to select a subset of functioning and symptom scale items for modeling. The second approach focused on global health and quality-of-life items based on a conceptual model. Model selection criteria included parsimony, statistical significance and logical consistency of parameter estimates, predictive accuracy, number of states described, and scale range. Results The optimal multiattribute model included nine variables for five items from different scales, described 144 unique states, predicted values ranging from 0.63 to 1.00, but it had poor predictive accuracy (cross-validation pseudo- R 2 = 0.056). The best-fitting global approach-based model described 24 unique states using eight indicators for two items from one scale (plus a constant) and predicted values ranging from 0.17 to 1.00 (cross-validation pseudo- R 2 = 0.127). Conclusions Multiattribute models produced a greater number of unique predicted values, while global models exhibited more desirable statistical properties and a wider range of values. The recommended models will enable users to predict cancer patients' utilities from existing and future QLQ-C30 data sets.

BACKGROUND: In a simulation using continuous data, we compared the performance of DeLong's test for nonparametric receiver operating characteristic (ROC) curves (Biometrics 1988; 44: 837–845) with that of Vuong's test for model selection (Econometrica 1989; 57: 307–333). Both tests were found to yield similar results regardless of sample size. Diagnostic tests are often measured on an ordinal rating scale, and nonparametric methods tend to underestimate the area under the ROC curve (AUC) when used with discrete data. Thus, it was conceivable that level of measurement might influence the performance of DeLong's test. OBJECTIVE: A second Monte Carlo simulation was performed to determine whether DeLong's and Vuong's tests behave differently when used with discrete data. METHODS: One thousand observations were randomly generated for a Bernoulli dependent variable and 11 binomial independent variables. The independents were generated such that realizations were integers ranging from 1–10. Bootstrapped estimates for AUC and logistic regression model log-likelihood (LL) were derived using 1000 replications of sample size 10, 25, 50, 100, 175, 250, and 500. At each sample size, predictors were compared on the basis of AUC using DeLong's test and model LL using Vuong's test. The random number seed was set so that identical samples were compared with each test. RESULTS: In general, the two tests yielded similar statistical conclusions. Asymptotically, the observed power of Vuong's test was greater than that of DeLong's test. In smaller samples, however, Vuong's test was slightly less powerful. The results of the two tests diverged in only three cases in small samples. CONCLUSIONS: The results of this analysis correspond to those of our simulation using continuous data. Though slightly less powerful than DeLong's test, Vuong's test is more flexible and is less time consuming. Given the results of both simulations, Vuong's test appears to present a useful alternative to ROC analysis for comparing the accuracy of diagnostic tests.