Search

BackgroundTreatment with phosphodiesterase type 5 inhibitors (PDE-5is) is effective in treating erectile dysfunction (ED).AimThe objective of this study was to determine the effect of PDE-5is on the incidence of major adverse cardiovascular (CV) events (MACE; composite outcome of CV death, hospitalization for myocardial infarction, coronary revascularization, stroke, heart failure, and unstable angina pectoris) and overall mortality.MethodsA retrospective observational cohort study was conducted in a large US claims database in men with ≥1 diagnosis of ED without prior MACE within 1 year, from January 1, 2006, to October 31, 2020. The exposed group had ≥1 claim for PDE-5i and the unexposed group had no claims for PDE-5i, and the groups were matched up to 1:4 on baseline risk variables.OutcomeThe primary outcome was MACE and the secondary outcomes were overall mortality and individual components of MACE, determined by multivariable Cox proportional hazard modeling.ResultsMatched plus multivariable analyses showed that MACE was lower by 13% in men exposed (n = 23 816) to PDE-5is (hazard ratio [HR] 0.87; 95% CI 0.79-0.95; P = .001) vs nonexposure (n = 48 682) over mean follow-up periods of 37 and 29 months, respectively, with lower incidence of coronary revascularization (HR 0.85; 95% CI 0.73-0.98; P = .029), heart failure (HR 0.83; 95% CI 0.72-0.97; P = .016), unstable angina (HR 0.78; 95% CI 0.64-0.96; P = .021), and CV death (HR 0.61; 95% CI 0.41-0.90; P = .014) with PDE-5i exposure. Phosphodiesterase type 5 inhibitor–exposed men had a 25% lower incidence of overall mortality (HR 0.75; 95% CI 0.65-0.87; P Clinical ImplicationsPDE-5is may have cardioprotective effects.Strengths and LimitationsStrengths are the large numbers of participants and consistency of the data; limitations include the retrospective nature of the study and unknown confounders.ConclusionsIn a large population of US men with ED, PDE-5i exposure was associated with lower incidence of MACE, CV death, and overall mortality risk compared to non-exposure. Risk reduction correlated with PDE-5i exposure level.

BACKGROUND The mission of the National Cancer Institute's Early Detection Research Network (EDRN) is to identify and validate cancer biomarkers for clinical use. Since its inception, EDRN investigators have learned a great deal about the process of validating biomarkers for clinical use. Translational research requires a broad spectrum of research expertise, and coordinating collaborative activities can be challenging. The EDRN has developed a robust triage and validation system that serves the roles of both “facilitator” and “brake.” CONTENT The system consists of (a) establishing a reference set of specimens collected under PRoBE (Prospective Specimen Collection Retrospective Blinded Evaluation) design criteria; (b) using the reference set to prevalidate candidate biomarkers before committing to full-scale validation; (c) performing full-scale validation for those markers that pass prevalidation testing; and (d) ensuring that the reference set is sufficiently large in numbers and volumes of sample that it can also be used to study future candidate biomarkers. This system provides rigorous and efficient evaluation of candidate biomarkers and biomarker panels. Reference sets should also be constructed to enable high-quality biomarker-discovery research. SUMMARY We describe the process of establishing our system in the hope that it will serve as an example of how to validate biomarkers for clinical application. We also hope that this description of the biospecimen reference sets available from the EDRN will encourage the biomarker research community—from academia or industry—to use this resource to advance biomarkers into clinical use.

Recent scientific and technological innovations have produced an abundance of potential markers that are being investigated for their use in disease screening and diagnosis. In evaluating these markers, it is often necessary to account for covariates associated with the marker of interest. Covariates may include subject characteristics, expertise of the test operator, test procedures or aspects of specimen handling. In this paper, we propose the covariate-adjusted receiver operating characteristic curve, a measure of covariate-adjusted classification accuracy. Nonparametric and semiparametric estimators are proposed, asymptotic distribution theory is provided and finite sample performance is investigated. For illustration we characterize the age-adjusted discriminatory accuracy of prostate-specific antigen as a biomarker for prostate cancer. Copyright 2009, Oxford University Press.

Consider a set of baseline predictors X to predict a binary outcome D and let Y be a novel marker or predictor. This paper is concerned with evaluating the performance of the augmented risk model P(D = 1|Y,X) compared with the baseline model P(D = 1|X). The diagnostic likelihood ratio, DLR(X)(y), quantifies the change in risk obtained with knowledge of Y = y for a subject with baseline risk factors X. The notion is commonly used in clinical medicine to quantify the increment in risk prediction due to Y. It is contrasted here with the notion of covariate-adjusted effect of Y in the augmented risk model. We also propose methods for making inference about DLR(X)(y). Case-control study designs are accommodated. The methods provide a mechanism to investigate if the predictive information in Y varies with baseline covariates. In addition, we show that when combined with a baseline risk model and information about the population distribution of Y given X, covariate-specific predictiveness curves can be estimated. These curves are useful to an individual in deciding if ascertainment of Y is likely to be informative or not for him. We illustrate with data from 2 studies: one is a study of the performance of hearing screening tests for infants, and the other concerns the value of serum creatinine in diagnosing renal artery stenosis.

There are two popular statistical approaches to biomarker evaluation. One models the risk of disease (or disease outcome) with, for example, logistic regression. A marker is considered useful if it has a strong effect on risk. The second evaluates classification performance by use of measures such as sensitivity, specificity, predictive values, and receiver operating characteristic curves. There is controversy about which approach is more appropriate. Moreover, the two approaches can give contradictory results on the same data. The authors present a new graphic, the predictiveness curve, which complements the risk modeling approach. It assesses the usefulness of a risk model when applied to the population. Although the predictiveness curve relates to classification performance measures, it also displays essential information about risk that is not displayed by the receiver operating characteristic curve. The authors propose that the predictiveness and classification performance of a marker, displayed together in an integrated plot, provide a comprehensive and cohesive assessment of a risk marker or model. The methods are demonstrated with data on prostate-specific antigen and risk factors from the Prostate Cancer Prevention Trial, 1993-2003.

Latent class analysis is used to assess diagnostic test accuracy when a gold standard assessment of disease is not available but results of multiple imperfect tests are. We consider the simplest setting, where 3 tests are observed and conditional independence (CI) is assumed. Closed-form expressions for maximum likelihood parameter estimates are derived. They show explicitly how observed 2- and 3-way associations between test results are used to infer disease prevalence and test true- and false-positive rates. Although interesting and reasonable under CI, the estimators clearly have no basis when it fails. Intuition for bias induced by conditional dependence follows from the analytic expressions. Further intuition derives from an Expectation Maximization (EM) approach to calculating the estimates. We discuss implications of our results and related work for settings where more than 3 tests are available. We conclude that careful justification of assumptions about the dependence between tests in diseased and nondiseased subjects is necessary in order to ensure unbiased estimates of prevalence and test operating characteristics and to provide these estimates clinical interpretations. Such justification must be based in part on a clear clinical definition of disease and biological knowledge about mechanisms giving rise to test results.

The case-control design is frequently used to study the discriminatory accuracy of a screening or diagnostic biomarker. Yet, the appropriate ratio in which to sample cases and controls has never been determined. It is common for researchers to sample equal numbers of cases and controls, a strategy that can be optimal for studies of association. However, considerations are quite different when the biomarker is to be used for classification. In this paper, we provide an expression for the optimal case-control ratio, when the accuracy of the biomarker is quantified by the receiver operating characteristic (ROC) curve. We show how it can be integrated with choosing the overall sample size to yield an efficient study design with specified power and type-I error. We also derive the optimal case-control ratios for estimating the area under the ROC curve and the area under part of the ROC curve. Our methods are applied to a study of a new marker for adenocarcinoma in patients with Barrett's esophagus.

Summary In studies to assess the accuracy of a screening test, often definitive disease assessment is too invasive or expensive to be ascertained on all the study subjects. Although it may be more ethical or cost effective to ascertain the true disease status with a higher rate in study subjects where the screening test or additional information is suggestive of disease, estimates of accuracy can be biased in a study with such a design. This bias is known as verification bias. Verification bias correction methods that accommodate screening tests with binary or ordinal responses have been developed; however, no verification bias correction methods exist for tests with continuous results. We propose and compare imputation and reweighting bias-corrected estimators of true and false positive rates, receiver operating characteristic curves and area under the receiver operating characteristic curve for continuous tests. Distribution theory and simulation studies are used to compare the proposed estimators with respect to bias, relative efficiency and robustness to model misspecification. The bias correction estimators proposed are applied to data from a study of screening tests for neonatal hearing loss.

A marker strongly associated with outcome (or disease) is often assumed to be effective for classifying persons according to their current or future outcome. However, for this assumption to be true, the associated odds ratio must be of a magnitude rarely seen in epidemiologic studies. In this paper, an illustration of the relation between odds ratios and receiver operating characteristic curves shows, for example, that a marker with an odds ratio of as high as 3 is in fact a very poor classification tool. If a marker identifies 10% of controls as positive (false positives) and has an odds ratio of 3, then it will correctly identify only 25% of cases as positive (true positives). The authors illustrate that a single measure of association such as an odds ratio does not meaningfully describe a marker's ability to classify subjects. Appropriate statistical methods for assessing and reporting the classification power of a marker are described. In addition, the serious pitfalls of using more traditional methods based on parameters in logistic regression models are illustrated.

Receiver operating characteristic (ROC) regression methodology is used to identify factors that affect the accuracy of medical diagnostic tests. In this paper, we consider a ROC model for which the ROC curve is a parametric function of covariates but distributions of the diagnostic test results are not specified. Covariates can be either common to all subjects or specific to those with disease. We propose a new estimation procedure based on binary indicators defined by the test result for a diseased subject exceeding various specified quantiles of the distribution of test results from non-diseased subjects with the same covariate values. This procedure is conceptually and computationally simplified relative to existing procedures. Simulation study results indicate that the approach has fairly high statistical efficiency. The new ROC regression methodology is used to evaluate childhood measurements of body mass index as a predictive marker of adult obesity.

Methods Using a population dataset of 10 000 individuals with an event rate of 10.2%, in which four biomarkers have no predictive ability, we repeatedly simulated studies and calculated the chance that the NRI statistic provides a positive statistically significant result. Subjects for training data (n = 420) and test data (n = 420 or 840) were randomly selected from the population, and corresponding NRI statistics and P values were calculated. For comparison, the change in the area under the receiver operating characteristic curve and likelihood ratio statistics were calculated. Results We found that rates of false-positive conclusions based on the NRI statistic were unacceptably high, being 63.0% in the training datasets and 18.8% to 34.4% in the test datasets. False-positive conclusions were rare when using the change in the area under the curve and occurred at the expected rate of approximately 5.0% with the likelihood ratio statistic.

Summary Estimating equations which are not necessarily likelihood-based score equations are becoming increasingly popular for estimating regression model parameters. This paper is concerned with estimation based on general estimating equations when true covariate data are missing for all the study subjects, but surrogate or mismeasured covariates are available instead. The method is motivated by the covariate measurement error problem in marginal or partly conditional regression of longitudinal data. We propose to base estimation on the expectation of the complete data estimating equation conditioned on available data. The regression parameters and other nuisance parameters are estimated simultaneously by solving the resulting estimating equations. The expected estimating equation (EEE) estimator is equal to the maximum likelihood estimator if the complete data scores are likelihood scores and conditioning is with respect to all the available data. A pseudo-EEE estimator, which requires less computation, is also investigated. Asymptotic distribution theory is derived. Small sample simulations are conducted when the error process is an order 1 autoregressive model. Regression calibration is extended to this setting and compared with the EEE approach. We demonstrate the methods on data from a longitudinal study of the relationship between childhood growth and adult obesity.

SUMMARY The appropriate interpretation of measurements often requires standardization for concomitant factors. For example, standardization of weight for both height and age is important in obesity research and in failure-to-thrive research in children. Regression quantiles from a reference population afford one intuitive and popular approach to standardization. Current methods for the estimation of regression quantiles can be classified as nonparametric with respect to distributional assumptions or as fully parametric. We propose a semiparametric method where we model the mean and variance as flexible regression spline functions and allow the unspecified distribution to vary smoothly as a function of covariates. Similarly to Cole and Green, our approach provides separate estimates and summaries for location, scale and distribution. However, similarly to Koenker and Bassett, we do not assume any parametric form for the distribution. Estimation for either cross-sectional or longitudinal samples is obtained by using estimating equations for the location and scale functions and through local kernel smoothing of the empirical distribution function for standardized residuals. Using this technique with data on weight, height and age for females under 3 years of age, we find that there is a close relationship between quantiles of weight for height and age and quantiles of body mass index (BMI=weight/height2) for age in this cohort.

SUMMARY Receiver operating characteristic curves (ROC's) are used to evaluate diagnostic tests when test results are not binary. They describe the inherent capacity of the test for distinguishing between truly diseased and nondiseased subjects. Although methodology for estimating and for comparing ROC'S is well developed, to date no general framework exists for evaluating covariate effects on ROC'S. We formulate a general regression model which allows the effects of covariates on test accuracy to be succinctly summarised. Such covariates might include, for example, characteristics of the patient or test environment, test type or severity of disease. The regression models are shown to arise naturally from some classic models for continuous or ordinal test data. Regression parameters are fitted using an estimating equation approach. The method is illustrated on data from a study of multiformat photographic images used for scintigraphy.

SUMMARY We consider the problem of missing covariate data in the context of censored failure time relative risk regression. Auxiliary covariate data, which are considered informative about the missing data but which are not explicitly part of the relative risk regression model, may be available. Full covariate information is available for a validation set. An estimated partial likelihood method is proposed for estimating relative risk parameters. This method is an extension of the estimated likelihood regression analysis method for uncensored data (Pepe, 1992; Pepe & Fleming, 1991). A key feature of the method is that it is nonparametric with respect to the association between the missing and observed, including auxiliary, covariate components. Asymptotic distribution theory is derived for the proposed estimated partial likelihood estimator in the case where the auxiliary or mismeasured covariates are categorical. Asymptotic efficiencies are calculated for exponential failure times using an exponential relative risk model. The estimated partial likelihood estimator compares favourably with a fully parametric maximum likelihood analysis. Comparisons are also made with a standard partial likelihood analysis which ignores the incomplete observations. Important efficiency gains can be made with the estimated partial likelihood method. Small sample properties are investigated through simulation studies.

SUMMARY We consider regression analysis when incomplete or auxiliary covariate data are available for all study subjects and, in addition, for a subset called the validation sample, true covariate data of interest have been ascertained. The term auxiliary data refers to data not in the regression model, but thought to be informative about the true missing covariate data of interest. We discuss a method which is nonparametric with respect to the association between available and missing data, allows missingness to depend on available response and covariate values, and is applicable to both cohort and case-control study designs. The method previously proposed by Flanders & Greenland (1991) and by Zhao & Lipsitz (1992) is generalised and asymptotic theory is derived. Our expression for the asymptotic variance of the estimator provides intuition regarding performance of the method. Optimal sampling strategies for the validation set are also suggested by the asymptotic results.

SUMMARY In the context of estimating ,3 from the regression model P ( YI X), relating response Y to covariates X, suppose that only a surrogate response S is available for most study subjects. Suppose that for a random subsample of the study cohort, termed the validation sample, the true outcome Y is available in addition to S. We consider maximum likelihood estimation of ,3 from such data and show that it is nonrobust to misspecification of the distribution relating the surrogate to the true outcome, P(S I Y, X). An alternative semiparametric method is also considered, which is nonparametric with respect to P(S I Y, X). Large-sample distribution theory for maximum estimated likelihood estimates is developed. An illustrative example is presented.

An ELISA was used to study long-term immunity and immunization responses to tetanus toxoid in 48 bone marrow transplant recipients. Among patients who were seropositive to tetanus before transplant, 51% had lost their seropositivity 1 year later. All patients who were not reimmunized with tetanus toxoid were seronegative 2 years after transplant. All patients who were seronegative before transplant remained seronegative 1 year later regardless of the donor's serologic status. There was no difference in the ability to remain seropositive to tetanus toxoid between patients with and without chronic graft-versus-host disease. Of 21 patients immunized with one dose of tetanus toxoid 1 year after transplant, 14 were seronegative at the time of immunization (response rate, 64%). At 1 year after immunization, 7 remained seropositive. Ten patients were reimmunized with two doses of tetanus toxoid. All responded and 90% remained seropositive 1 year later. When 21 patients were primarily immunized with three doses of tetanus toxoid, all patients seronegative at immunization responded and all tested patients remained seropositive 2 years later. The immunization responses were significantly superior in patients receiving three doses compared with those who received one. Reimmunization with tetanus toxoid of long-term survivors after marrow transplant seems necessary. A three-dose immunization schedule is recommended to obtain an adequate immune response.

Developing biomarkers that can predict whether patients are likely to benefit from an intervention is a pressing objective in many areas of medicine. Recent guidance documents have recommended that the accuracy of predictive biomarkers, ie, sensitivity, specificity, and positive and negative predictive values, should be assessed. We clarify the meanings of these entities for predictive markers and demonstrate that generally they cannot be estimated from data without making strong untestable assumptions. Language suggesting that predictive biomarkers can identify patients who benefit from an intervention is also widespread. We show that in general one cannot estimate the chance that a patient will benefit from treatment. We recommend instead that predictive biomarkers be evaluated with respect to their ability to predict clinical outcomes among patients treated and among patients receiving standard of care, and the population impact of treatment rules based on those predictions. Ideally these entities are estimated from a randomized trial comparing the experimental intervention with standard of care.

Khalili et al. (1) evaluated the clinical utility of the Framingham cardiovascular risk model in a Middle Eastern cohort study. We commend the authors for their use of highly relevant measures of model performance, the net benefit (2, 3) and net benefit fraction (4, 5). However, we would like to make several points regarding their interpretation and reporting of these measures. Interpretation of the net benefit and net benefit fraction The population net benefit (NB) of a risk model, used to classify individuals as being at high 10-year risk of cardiovascular disease using the risk threshold rH, is (1) where B is the benefit of classifying as high-risk a subject who would develop cardiovascular disease absent treatment (a “cardiovascular event”), H is the harm of classifying as high-risk a subject who would not develop cardiovascular disease absent treatment (a “cardiovascular nonevent”), TPR(rH) and FPR(rH) are the true- and false-positive rates associated with this risk threshold, and ρ is the proportion of subjects who experience cardiovascular events. Vickers and Elkin (2) set B to 1, so that the units are true positive classifications, and use the optimal risk threshold rH = H/H + B (6) in order to rewrite equation 1 as (2) Baker et al. (4, 5) scale equation 2 by ρ, which is the maximum possible net benefit for a model that classifies as high-risk all cardiovascular events and all others as low-risk. The net benefit fraction (NBF) is therefore (3) Khalili et al. interpret NBF(0.2) = 0.156 for the Framingham risk model applied to women as “the fraction of the incidence rate that could be predicted and prevented” (1, p. 178). However, this is not a valid interpretation. Their interpretation seems to relate to the TPR of the model, whereas NBF(0.2) = 0.156 is more accurately interpreted as a discounted TPR; the TPR is offset by the FPR where false-positives are put on the same scale as true-positives (7). We prefer to interpret NBF(0.2) = 0.156 as the same benefit that would be achieved by classifying as high-risk 15.6% of cardiovascular events and none of the nonevents (2, 3). The authors' interpretation also suggests that all cardiovascular events classified as high-risk would necessarily be prevented; however, this would depend on the efficacy of subsequent interventions. Net benefit for different subsets of the population Net benefit curves, which plot the net benefit in equation 2 versus the risk threshold rH (2, 3), are also used by Khalili et al. The authors recommend these curves for examining the net benefit for a range of risk thresholds, given that different patient subgroups may have different side effects of interventions and therefore different risk thresholds. However, the net benefit curve shows the total population net benefit at each risk threshold. Two points on the curve cannot be interpreted as the net benefits for 2 different subpopulations because, if the risk distribution in the subpopulation differs from the risk distribution in the whole population, the net benefit in the subpopulation is different from the net benefit in the whole population at that threshold. For example, men and women in the Middle Eastern cohort have different values for TPR, FPR, and ρ and therefore have net benefits that differ from the population net benefit shown on the net benefit curve. Recommendation We have several practical recommendations. The first is to stratify any analysis by important patient characteristics that affect risk threshold—for example, gender as in the study by Khalili et al. (1). The second is to report the net benefit fraction together with its constituents, the TPR and FPR, over a range of risk thresholds. Seeing these individual components helps in digesting the net benefit. Knowing the TPR and FPR is also helpful when risk thresholds are chosen “irrationally.” That is, the net benefit in equation 2 assumes that the risk threshold reflects the cost-benefit ratio, rH = H/H + B. However, individuals or policy-makers may choose risk thresholds in ways that do not reflect the cost-benefit ratio. For example, in their commentary accompanying Khalili et al.'s article, D'Agostino and Pencina (8) suggest that the risk threshold rH = 0.2 was chosen without regard to cost-benefit considerations. Reporting the TPR and FPR as a function of risk threshold allows the reader to choose a threshold and to perform any calculus desired to take into consideration both components of model performance.