Search

RESEARCH OBJECTIVE: Prescribing opioids, particularly the number of pills, is associated with greater likelihood of patients developing longer‐term opioid dependence. Nudges targeted to clinicians are a low‐cost strategy that could reduce unnecessary opioid prescribing. In particular, clinician‐focused peer comparison feedback has been effective in influencing prescribing for other medications. However, peer comparison feedback has not been well‐tested for opioid prescribing, alone or compared against other feedback approaches such as individual audit feedback. STUDY DESIGN: We conducted a pragmatic, four‐arm factorial cluster randomized clinical trial among 48 emergency department (ED) and urgent care (UC) practice sites within Sutter Health System including a 6‐month pre‐intervention period and a 6‐month intervention. 438 clinicians were cluster randomized by practice site. Interventions were delivered to clinicians monthly by email. Peer comparison feedback included the number of pills per opioid prescription and proportion of encounters with an opioid prescription in the prior 3 months, relative to that of their practice site peers. Individual audit feedback included the number of prescriptions with >30 pills in the prior 3 months, indicating potential outlier prescriptions. The primary outcome measure was the change in the number of pills per prescription. Secondary outcomes included changes in the proportion of pills and milligrams of morphine equivalents (MME) per prescription. POPULATION STUDIED: The sample included 263 ED and 175 UC clinicians comprised of 294,962 patient encounters, with a mean (SD) age of 49 years (19), 56% female, 9% Black, and 21% Hispanic. PRINCIPAL FINDINGS: During the pre‐intervention period, there was a mean (SD) of 15.1 pills (3.9) and 76.3 MME (23.0) per prescription, and 9.3% of encounters with an opioid prescription. In adjusted analyses compared to usual care during the intervention, there was a significant decrease in pills per prescription among clinicians receiving peer comparison feedback alone (−0.9 pills per prescription; 95% CI ‐1.5 to −0.3, P = 0.002) and receiving both peer comparison and individual audit feedback (−1.4 pills per prescription; 95% CI ‐2 to −0.8, P

Importance Statin therapy is underused for many patients who could benefit. Objective To evaluate the effect of passive choice and active choice interventions in the electronic health record (EHR) to promote guideline-directed statin therapy. Design, setting, and participants Three-arm randomized clinical trial with a 6-month preintervention period and 6-month intervention. Randomization conducted at the cardiologist level at 16 cardiology practices in Pennsylvania and New Jersey. The study included 82 cardiologists and 11 693 patients. Data were analyzed between May 8, 2019, and January 9, 2020. Interventions In passive choice, cardiologists had to manually access an alert embedded in the EHR to select options to initiate or increase statin therapy. In active choice, an interruptive EHR alert prompted the cardiologist to accept or decline guideline-directed statin therapy. Cardiologists in the control group were informed of the trial but received no other interventions. Main outcomes and measures Primary outcome was statin therapy at optimal dose based on clinical guidelines. Secondary outcome was statin therapy at any dose. Results The sample comprised 11 693 patients with a mean (SD) age of 63.8 (9.1) years; 58% were male (n = 6749 of 11 693), 66% were White (n = 7683 of 11 693), and 24% were Black (n = 2824 of 11 693). The mean (SD) 10-year atherosclerotic cardiovascular disease (ASCVD) risk score was 15.4 (10.0); 68% had an ASVCD clinical diagnosis. Baseline statin prescribing rates at the optimal dose were 40.3% in the control arm, 39.1% in the passive choice arm, and 41.2% in the active choice arm. In adjusted analyses, the change in statin prescribing rates at optimal dose over time was not significantly different from control for passive choice (adjusted difference in percentage points, 0.2; 95% CI, -2.9 to 2.8; P = .86) or active choice (adjusted difference in percentage points, 2.4; 95% CI, -0.6 to 5.0; P = .08). In adjusted analyses of the subset of patients with clinical ASCVD, the active choice intervention resulted in a significant increase in statin prescribing at optimal dose relative to control (adjusted difference in percentage points, 3.8; 95% CI, 1.0-6.4; P = .008). No other subset analyses were significant. There were no significant changes in statin prescribing at any dose for either intervention. Conclusions and relevance The passive choice and active choice interventions did not change statin prescribing. In the subgroup of patients with clinical ASCVD, the active choice intervention led to a small increase in statin prescribing at the optimal dose, which could inform the design or targeting of future interventions. Trial registration ClinicalTrials.gov Identifier: NCT03271931.

Importance Diagnostic variation among pathologists interpreting cutaneous melanocytic lesions could lead to suboptimal care. Objective To estimate the potential association of second-opinion strategies in the histopathologic diagnosis of cutaneous melanocytic lesions with diagnostic accuracy and 1-year population-level costs in the US. Design, Setting, and Participants Decision analysis with 1-year time horizon including melanocytic lesion diagnoses available from US pathologists participating in the Melanoma Pathology Study (M-Path) and from the study panel of reference pathologists who classified cases using the MPATH-Dx classification tool. M-Path data collection occurred from July 2013 through March 2015; analyses for the present study were performed between April 2015 and January 2021. Exposures Various second-opinion strategies for interpretation of melanocytic cutaneous lesions. Main Outcomes and Measures Estimated accuracy of pathologists’ diagnoses, defined as concordance with the reference panel diagnoses, and 1-year postbiopsy medical costs under various second-opinion strategies. Expected percentage of concordant diagnoses, including percentages of overinterpretation and underinterpretation, and 1-year costs of medical care per 100 000 in the US population. Results Decision-analytic model parameters were based on diagnostic interpretations for 240 cases by 187 pathologists compared with reference panel diagnoses. Without second opinions, 83.2% of diagnoses in the US were estimated to be accurate—ie, concordant with the reference diagnosis; with overinterpretation (8.0%) or underinterpretation (8.8%), and 16 850 misclassified diagnoses per 100 000 biopsies. Accuracy increased under all second-opinion strategies. Accuracy (87.4% concordance with 3.6% overinterpretation and 9.1% underinterpretation) and cost (an increase of more than $10 million per 100 000 biopsies per year) were highest when second opinions were universal (eg, performed on all biopsies), relative to no second opinions. A selective second-opinion strategy based on pathologists’ desire or institutional requirements for a second opinion was most accurate (86.5% concordance; 4.4% overinterpretation; 9.1% underinterpretation) and would reduce costs by more than $1.9 million per 100 000 skin biopsies relative to no second opinions. Improvements in diagnostic accuracy with all second-opinion strategies were associated with reductions in overinterpretation but not underinterpretation. Conclusions and Relevance In this decision-analytic model, selective second-opinion strategies for interpretation of melanocytic skin lesions showed the potential to improve diagnostic accuracy and decrease costs relative to no second opinions or universal second opinions.

Key Points Question Can different forms of financial incentives (unconditional, conditional, or lottery) boost response rates to mailed colorectal cancer screening outreach? Findings In this randomized clinical trial of 897 patients, there was no statistically significant difference in screening response rates at 2 and 6 months between the incentive arms and mailed outreach without incentive. Meaning Different forms of financial incentives of the same expected value ($10) did not increase fecal immunochemical test completion rates, as the incentive value have been too small or financial incentives may not be effective in this context., Importance Mailing fecal immunochemical test (FIT) kits to patients’ homes has been shown to boost colorectal cancer screening rates, but response rates remain limited, and organized programs typically require repeated outreach attempts. Behavioral economics has shown that offering salient financial incentives to patients may increase participation in preventive health. Objective To compare the impact of different financial incentives for mailed FIT outreach. Design, Setting, and Participants This 4-parallel-arm randomized clinical trial included patients aged 50 to 75 years who had an established primary care clinician, at least 2 visits in the prior 2 years, and were eligible for colorectal cancer screening and not up-to-date. This study was conducted at urban primary care practices at an academic health system from December 2015 to February 2018. Data analysis was conducted from March 2018 to September 2018. Interventions Eligible patients received a letter from their primary care clinician that included a mailed FIT kit and instructions for use. They were randomized in a 1:1:1:1 ratio to receive (1) no financial incentive; (2) an unconditional $10 incentive included with the mailing; (3) a $10 incentive conditional on FIT completion; or (4) a conditional lottery with a 1-in-10 chance of winning $100 after FIT completion. Main Outcomes and Measures Fecal immunochemical test completion within 2 and 6 months of initial outreach. Results A total of 897 participants were randomized, with a median age of 57 years (interquartile range, 52-62 years); 56% were women, and 69% were black. The overall completion rate across all arms was 23.5% at 2 months. The completion rate at 2 months was 26.0% (95% CI, 20.4%-32.3%) in the no incentive arm, 27.2% (95% CI, 21.5%-33.6%) in the unconditional incentive arm, 23.2% (95% CI, 17.9%-29.3%) in the conditional incentive arm, and 17.7% (95% CI, 13.0%-23.3%) in the lottery incentive arm. None of the arms with an incentive were statistically superior to the arm without incentive. The overall FIT completion rate across all arms was 28.9% at 6 months, and there was also no difference by arm. The completion rate at 6 months was 32.7% (95% CI, 26.6%-39.3%) in the no incentive arm, 31.7% (95% CI, 25.7%-38.2%) in the unconditional incentive arm, 26.8% (95% CI, 21.1%-33.1%) in the conditional incentive arm, and 24.3% (95% CI, 18.9%-30.5%) in the lottery incentive arm. Conclusions and Relevance Mailed FIT resulted in high colorectal cancer screening response rates in this population, but different forms of financial incentives of the same expected value ($10) did not incrementally increase FIT completion rates. The incentive value may have been too small or financial incentives may not be effective in this context. Trial Registration ClinicalTrials.gov Identifier: NCT02594150, This 4-parallel-arm randomized clinical trial determines the effectiveness of financial incentives on fecal immunochemical test screening completion vs no incentive among adults who were sent kits by mail.

Surgeons may receive a different diagnosis when a breast biopsy is interpreted by a second pathologist. The extent to which diagnostic agreement by the same pathologist varies at two time points is unknown. Pathologists from eight U.S. states independently interpreted 60 breast specimens, one glass slide per case, on two occasions separated by ≥9 months. Reproducibility was assessed by comparing interpretations between the two time points; associations between reproducibility (intraobserver agreement rates); and characteristics of pathologists and cases were determined and also compared with interobserver agreement of baseline interpretations. Sixty-five percent of invited, responding pathologists were eligible and consented; 49 interpreted glass slides in both study phases, resulting in 2940 interpretations. Intraobserver agreement rates between the two phases were 92% [95% confidence interval (CI) 88–95] for invasive breast cancer, 84% (95% CI 81–87) for ductal carcinoma-in-situ, 53% (95% CI 47–59) for atypia, and 84% (95% CI 81–86) for benign without atypia. When comparing all study participants’ case interpretations at baseline, interobserver agreement rates were 89% (95% CI 84–92) for invasive cancer, 79% (95% CI 76–81) for ductal carcinoma-in-situ, 43% (95% CI 41–45) for atypia, and 77% (95% CI 74–79) for benign without atypia. Interpretive agreement between two time points by the same individual pathologist was low for atypia and was similar to observed rates of agreement for atypia between different pathologists. Physicians and patients should be aware of the diagnostic challenges associated with a breast biopsy diagnosis of atypia when considering treatment and surveillance decisions.

Background: Many circulating biomarkers have been reported for the diagnosis of breast cancer, but few, if any, have undergone rigorous credentialing using prospective cohorts and blinded evaluation. Methods: The NCI Early Detection Research Network (EDRN) has created a prospective, multicenter collection of plasma and serum samples from 832 subjects designed to evaluate circulating biomarkers for the detection and diagnosis of breast cancer. These samples are available to investigators who wish to evaluate their biomarkers using a set of blinded samples. The breast cancer reference set is composed of blood samples collected using a standard operating procedure at four U.S. medical centers from 2008 to 2010 from women undergoing either tissue diagnosis for breast cancer or routine screening mammography. The reference set contains samples from women with incident invasive cancer (n = 190), carcinoma in situ (n = 55), benign pathology with atypia (n = 63), benign disease with no atypia (n = 231), and women with no evidence of breast disease by screening mammography (BI-RADS 1 or 2, n = 276). Using a subset of plasma samples (n = 505) from the reference set, we analyzed 90 proteins by multiplexed immunoassays for their potential utility as diagnostic markers. Results: We found that none of these markers is useful for distinguishing cancer from benign controls. However, elevated CA-125 does appear to be a candidate marker for estrogen receptor–negative cancers. Conclusions: Markers that can distinguish benign breast conditions from invasive cancer have not yet been found. Impact: Availability of prospectively collected samples should improve future validation efforts. Cancer Epidemiol Biomarkers Prev; 24(2); 435–41. ©2014 AACR.

Objective To quantify the accuracy and reproducibility of pathologists’ diagnoses of melanocytic skin lesions. Design Observer accuracy and reproducibility study. Setting 10 US states. Participants Skin biopsy cases (n=240), grouped into sets of 36 or 48. Pathologists from 10 US states were randomized to independently interpret the same set on two occasions (phases 1 and 2), at least eight months apart. Main outcome measures Pathologists’ interpretations were condensed into five classes: I (eg, nevus or mild atypia); II (eg, moderate atypia); III (eg, severe atypia or melanoma in situ); IV (eg, pathologic stage T1a (pT1a) early invasive melanoma); and V (eg, ≥pT1b invasive melanoma). Reproducibility was assessed by intraobserver and interobserver concordance rates, and accuracy by concordance with three reference diagnoses. Results In phase 1, 187 pathologists completed 8976 independent case interpretations resulting in an average of 10 (SD 4) different diagnostic terms applied to each case. Among pathologists interpreting the same cases in both phases, when pathologists diagnosed a case as class I or class V during phase 1, they gave the same diagnosis in phase 2 for the majority of cases (class I 76.7%; class V 82.6%). However, the intraobserver reproducibility was lower for cases interpreted as class II (35.2%), class III (59.5%), and class IV (63.2%). Average interobserver concordance rates were lower, but with similar trends. Accuracy using a consensus diagnosis of experienced pathologists as reference varied by class: I, 92% (95% confidence interval 90% to 94%); II, 25% (22% to 28%); III, 40% (37% to 44%); IV, 43% (39% to 46%); and V, 72% (69% to 75%). It is estimated that at a population level, 82.8% (81.0% to 84.5%) of melanocytic skin biopsy diagnoses would have their diagnosis verified if reviewed by a consensus reference panel of experienced pathologists, with 8.0% (6.2% to 9.9%) of cases overinterpreted by the initial pathologist and 9.2% (8.8% to 9.6%) underinterpreted. Conclusion Diagnoses spanning moderately dysplastic nevi to early stage invasive melanoma were neither reproducible nor accurate in this large study of pathologists in the USA. Efforts to improve clinical practice should include using a standardized classification system, acknowledging uncertainty in pathology reports, and developing tools such as molecular markers to support pathologists’ visual assessments.

PURPOSE: To estimate the potential near-term population impact of alternative second opinion breast biopsy pathology interpretation strategies. METHODS: Decision analysis examining 12-month outcomes of breast biopsy for nine breast pathology interpretation strategies in the U.S. health system. Diagnoses of 115 practicing pathologists in the Breast Pathology Study were compared to reference-standard-consensus diagnoses with and without second opinions. Interpretation strategies were defined by whether a second opinion was sought universally or selectively (e.g., 2nd opinion if invasive). Main outcomes were the expected proportion of concordant breast biopsy diagnoses, the proportion involving over-or under-interpretation, and cost of care in U.S. dollars within one-year of biopsy. RESULTS: Without a second opinion, 92.2% of biopsies received a concordant diagnosis. Concordance rates increased under all second opinion strategies, and the rate was highest (95.1%) and under-treatment lowest (2.6%) when all biopsies had second opinions. However, overtreatment was lowest when second opinions were sought selectively for initial diagnoses of invasive cancer, DCIS, or atypia (1.8 vs. 4.7% with no 2nd opinions). This strategy also had the lowest projected 12-month care costs ($5.907 billion vs. $6.049 billion with no 2nd opinions). CONCLUSIONS: Second opinion strategies could lower overall care costs while reducing both over-and under-treatment. The most accurate cost-saving strategy required second opinions for initial diagnoses of invasive cancer, DCIS, or atypia.

This diagnostic study evaluates whether fellowship training and board certification of physicians in dermatopathology affect the diagnostic reliability of second-opinion strategies for difficult-to-diagnose cutaneous melanocytic lesions., Key Points Question For cutaneous melanocytic lesions that are difficult to diagnose, do second opinions rendered by pathologists who have board certification and/or fellowship training in dermatopathology improve overall reliability of diagnosis? Findings In this diagnostic study of 240 melanocytic lesions from the Melanoma Pathology Study data set, misclassification of melanocytic lesions was lowest when first, second, and third consulting reviewers were subspecialty-trained dermatopathologists and when all lesions were subject to second opinions; misclassification was highest when reviewers were all general pathologists lacking subspecialty training. Variability of in situ and thin invasive melanoma was relatively intractable to all examined strategies. Meaning The findings suggest that second opinions rendered by dermatopathologists improve overall reliability of diagnosis of melanocytic lesions but do not eliminate or substantially reduce misclassification., Importance Histopathologic criteria have limited diagnostic reliability for a range of cutaneous melanocytic lesions. Objective To evaluate the association of second-opinion strategies by general pathologists and dermatopathologists with the overall reliability of diagnosis of difficult melanocytic lesions. Design, Setting, and Participants This diagnostic study used samples from the Melanoma Pathology Study, which comprises 240 melanocytic lesion samples selected from a dermatopathology laboratory in Bellevue, Washington, and represents the full spectrum of lesions from common nevi to invasive melanoma. Five sets of 48 samples were evaluated independently by 187 US pathologists from July 15, 2013, through May 23, 2016. Data analysis was performed from April 2016 through November 2017. Main Outcomes and Measures Accuracy of diagnosis, defined as concordance with an expert consensus diagnosis of 3 experienced pathologists, was assessed after applying 10 different second-opinion strategies. Results Among the 187 US pathologists examining the 24 lesion samples, 113 were general pathologists (65 men [57.5%]; mean age at survey, 53.7 years [range, 33.0-79.0 years]) and 74 were dermatopathologists (49 men [66.2%]; mean age at survey, 46.4 years [range, 33.0-77.0 years]). Among the 8976 initial case interpretations, physicians desired second opinions for 3899 (43.4%), most often for interpretation of severely dysplastic nevi. The overall misclassification rate was highest when interpretations did not include second opinions and initial reviewers were all general pathologists lacking subspecialty training (52.8%; 95% CI, 51.3%-54.3%). When considering different second opinion strategies, the misclassification of melanocytic lesions was lowest when the first, second, and third consulting reviewers were subspecialty-trained dermatopathologists and when all lesions were subject to second opinions (36.7%; 95% CI, 33.1%-40.7%). When the second opinion strategies were compared with single interpretations without second opinions, the reductions in misclassification rates for some of the strategies were statistically significant, but none of the strategies eliminated diagnostic misclassification. Melanocytic lesions in the middle of the diagnostic spectrum had the highest misclassification rates (eg, moderately or severely dysplastic nevus, Spitz nevus, melanoma in situ, and pathologic stage [p]T1a invasive melanoma). Variability of in situ and thin invasive melanoma was relatively intractable to all examined strategies. Conclusions and Relevance The results of this study suggest that second opinions rendered by dermatopathologists improve reliability of melanocytic lesion diagnosis. However, discordance among pathologists remained high.

Background Diagnostic agreement among pathologists is 84% for ductal carcinoma in situ (DCIS). Studies of interpretive variation according to grade are limited. Methods A national sample of 115 pathologists interpreted 240 breast pathology test set cases in the Breast Pathology Study and their interpretations were compared to expert consensus interpretations. We assessed agreement of pathologists' interpretations with a consensus reference diagnosis of DCIS dichotomised into low- and high-grade lesions. Generalised estimating equations were used in logistic regression models of rates of under- and over-interpretation of DCIS by grade. Results We evaluated 2097 independent interpretations of DCIS (512 low-grade DCIS and 1585 high-grade DCIS). Agreement with reference diagnoses was 46% (95% confidence interval [CI] 42–51) for low-grade DCIS and 83% (95% CI 81–86) for high-grade DCIS. The proportion of reference low-grade DCIS interpretations over-interpreted by pathologists (i.e. categorised as either high-grade DCIS or invasive cancer) was 23% (95% CI 19–28); 30% (95% CI 26–34) were interpreted as a lower diagnostic category (atypia or benign proliferative). Reference high-grade DCIS was under-interpreted in 14% (95% CI 12–16) of observations and only over-interpreted 3% (95% CI 2–4). Conclusion Grade is a major factor when examining pathologists' variability in diagnosing DCIS, with much lower agreement for low-grade DCIS cases compared to high-grade. These findings support the hypothesis that low-grade DCIS poses a greater interpretive challenge than high-grade DCIS, which should be considered when developing DCIS management strategies.

Importance Compared with white American (WA) women, African American (AA) women have a 2-fold higher incidence of breast cancers that are negative for estrogen receptor, progesterone receptor, andERBB2(triple-negative breast cancer [TNBC]). Triple-negative breast cancer, compared with non-TNBC, likely arises from different pathogenetic pathways, and benign breast disease (BBD) predicts future non-TNBC. Objective To determine whether AA identity remains associated with TNBC for women with a prior diagnosis of BBD. Design, Setting, and Participants This study is a retrospective analysis of data of a cohort of 2588 AA and 3566 WA women aged between 40 and 70 years with a biopsy-proven BBD diagnosis. The data—obtained from the Pathology Information System of Henry Ford Health System (HFHS), an integrated multihospital and multispecialty health care system headquartered in Detroit, Michigan—include specimens of biopsies performed between January 1, 1994, and December 31, 2005. Data analysis was performed from November 1, 2015, to June 15, 2016. Main Outcomes and Measures Subsequent breast cancer was stratified on the basis of combinations of hormone receptor andERBB2expression. Results Case management, follow-up, and outcomes received or obtained by our cohort of 2588 AA and 3566 WA patients were similar, demonstrating that HFHS delivered care equitably. Subsequent breast cancers developed in 103 (4.1%) of AA patients (mean follow-up interval of 6.8 years) and 143 (4.0%) of WA patients (mean follow-up interval of 6.1 years). More than three-quarters of subsequent breast cancers in each subset were ductal carcinoma in situ or stage I. The 10-year probability estimate for developing TNBC was 0.56% (95% CI, 0.32%-1.0%) for AA patients and 0.25% (95% CI, 0.12%-0.53%) for WA patients. Among the 66 AA patients who developed subsequent invasive breast cancer, 16 (24.2%) developed TNBC compared with 7 (7.4%) of the 94 WA patients who developed subsequent invasive breast cancers and had complete biomarker data (P = .01). Conclusions and Relevance This study is the largest analysis to date of TNBC in the context of racial/ethnic identity and BBD as risk factors. The study found that AA identity persisted as a significant risk factor for TNBC. This finding suggests that AA identity is associated with inherent susceptibility for TNBC pathogenetic pathways.

Background Diagnostic interpretations of melanocytic skin lesions vary widely among pathologists, yet the underlying reasons remain unclear. Objective Identify pathologist characteristics associated with rates of accuracy and reproducibility. Methods Pathologists independently interpreted the same set of biopsy specimens from melanocytic lesions on 2 occasions. Diagnoses were categorized into 1 of 5 classes according to the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis system. Reproducibility was determined by pathologists' concordance of diagnoses across 2 occasions. Accuracy was defined by concordance with a consensus reference standard. Associations of pathologist characteristics with reproducibility and accuracy were assessed individually and in multivariable logistic regression models. Results Rates of diagnostic reproducibility and accuracy were highest among pathologists with board certification and/or fellowship training in dermatopathology and in those with 5 or more years of experience. In addition, accuracy was high among pathologists with a higher proportion of melanocytic lesions in their caseload composition and higher volume of melanocytic lesions. Limitations Data gathered in a test set situation by using a classification tool not currently in clinical use. Conclusion Diagnoses are more accurate among pathologists with specialty training and those with more experience interpreting melanocytic lesions. These findings support the practice of referring difficult cases to more experienced pathologists to improve diagnostic accuracy, although the impact of these referrals on patient outcomes requires additional research.

IMPORTANCE The recently updated American Joint Committee on Cancer (AJCC) classification of cancer staging, the AJCC Cancer Staging Manual, 8th edition (AJCC 8), includes revisions to definitions of T1a vs T1b or greater. The Melanoma Pathology Study database affords a comparison,of pathologists’ concordance and reproducibility in the microstaging of melanoma according to both the existing 7th edition (AJCC 7) and the new AJCC 8. OBJECTIVE To compare AJCC 7 and AJCC 8 to examine whether changes to the definitions of T1a and T1b or greater are associated with changes in concordance and reproducibility. DESIGN, SETTING, AND PARTICIPANTS In this diagnostic study conducted as part of the national Melanoma Pathology Study across US states, 187 pathologists interpreting melanocytic skin lesions in practice completed 4342 independent case interpretations of 116 invasive melanoma cases. A consensus reference diagnosis and participating pathologists’ interpretations were classified into the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis class IV (T1a) or class V ( T1b) using both the AJCC 7 and AJCC 8 criteria. MAIN OUTCOMES AND MEASURES Concordance with consensus reference diagnosis, interobserver reproducibility, and intraobserver reproducibility. RESULTS For T1a diagnoses, participating pathologists’ concordance with the consensus reference diagnosis increased from 44% (95% CI, 41%−48%) to 54% (95% CI, 51%−57%) using AJCC 7 and AJCC 8 criteria, respectively. The concordance for cases of T1b or greater increased from 72% (95% CI, 69%−75%) to 78% (95% CI, 75%−80%). Intraobserver reproducibility of diagnoses also improved, increasing from 59% (95% CI, 56%−63%) to 64% (95% CI, 62%−67%) for T1a invasive melanoma, and from 74% (95% CI, 71%−76%) to 77% (95% CI, 74%−79%) for T1b or greater invasive melanoma cases. CONCLUSIONS AND RELEVANCE Melanoma staging in AJCC 8 shows greater reproducibility and higher concordance with a reference standard. Improved classification of invasive melanoma can be expected after implementation of AJCC 8, suggesting a positive impact on patients. However, despite improvement, concordance and reproducibility remain low.

With the emergence of new molecular biotechnologies, the intensity of biomarker research has increased greatly, yet the scientific rigor of biomarker research lags far behind that of therapeutic research ( 1 – 3 ). In therapeutic research, the randomized placebocontrolled, blinded clinical trial takes the central role in pivotal evaluation of a new therapy. Standards for the conduct of such studies have been agreed upon internationally ( 4 ) ( www.ich.org ), and books and journals have been devoted to nuances of their design and analysis. Analogously, in this commentary, we consider key aspects of design for pivotal evaluation of a biomarker ’ s capacity to correctly classify a subject’s outcome (ie, classification accuracy), in which a subject’s outcome may be his or her current disease status or a future event for him or her. We propose a prospective-specimencollection, retrospective-blinded-evaluation (PRoBE) design in which biologic specimens are collected prospectively from a cohort that represents the target population that is envisioned for clinical application of the biomarker. Specimens and clinical data are collected in the absence of knowledge about patient outcome. After outcome status is ascertained, case patients with the outcome and control subjects without it are selected randomly from the cohort and their specimens are assayed for the biomarker in a fashion that is blinded to case – control status. Although every biomarker study has its own special considerations, as does every randomized clinical trial, we propose that elucidation of the key design issues in this commentary will help move the field toward standards of practice. Biomarkers are developed for many different purposes, including for classifi cation and prediction, as surrogate outcomes in clinical trials, as measures of toxic or preventive exposures, or as a guide to individual treatment choice ( 5 ). In this commentary, we consider only the fi rst category, which includes diagnostic, screening, and prognostic markers. A diagnostic marker is used in people with signs or symptoms to aid in assessing whether they have a condition. A screening marker is used in asymptomatic people to detect a disease or condition at an early stage. A prognostic marker is used in subjects diagnosed with a condition to predict subsequent outcomes, such as disease recurrence or progression. The PRoBE design is intended for all of these applications. Development of a biomarker is a process that begins with biomarker discovery, is followed by rigorous evaluation of classifi cation accuracy, and then terminates with the evaluation of the impact of the biomarker on clinical outcomes ( 6 , 7 ). Multiple studies may be involved in each stage. For example, in the discovery stage, a sequence of studies may be performed to identify biomarkers from among a large pool of candidates, to validate these individual markers in independent samples, and to optimally combine markers from a panel. We focus on the intermediate stage, after all discovery work is completed. We assume that a well-defi ned biomarker, or marker combination, is to be defi nitively evaluated for its classifi cation accuracy in a specifi c clinical application. If it can be shown that the marker has acceptable classifi cation accuracy, the marker should move to the fi nal stage of evaluation, in which the net benefi t to patients is assessed by incorporating effects of clinical interventions recommended for patients on the basis of their biomarker results. Clearly, before biomarker results are used in making medical decisions for individuals, it is crucial to know how accurately the marker classifi es or predicts their outcomes.

There are two popular statistical approaches to biomarker evaluation. One models the risk of disease (or disease outcome) with, for example, logistic regression. A marker is considered useful if it has a strong effect on risk. The second evaluates classification performance by use of measures such as sensitivity, specificity, predictive values, and receiver operating characteristic curves. There is controversy about which approach is more appropriate. Moreover, the two approaches can give contradictory results on the same data. The authors present a new graphic, the predictiveness curve, which complements the risk modeling approach. It assesses the usefulness of a risk model when applied to the population. Although the predictiveness curve relates to classification performance measures, it also displays essential information about risk that is not displayed by the receiver operating characteristic curve. The authors propose that the predictiveness and classification performance of a marker, displayed together in an integrated plot, provide a comprehensive and cohesive assessment of a risk marker or model. The methods are demonstrated with data on prostate-specific antigen and risk factors from the Prostate Cancer Prevention Trial, 1993-2003.

BACKGROUND Many cancer biomarker research studies seek to develop markers that can accurately detect or predict future onset of disease. To design and evaluate these studies, one must specify the levels of accuracy sought. However, justified target levels are rarely available. METHODS We describe a way to calculate target levels of sensitivity and specificity for a biomarker intended to be applied in a defined clinical context. The calculation requires knowledge of the prevalence or incidence of cases in the clinical population and the ratio of benefit associated with the clinical consequences of a positive biomarker test in cases (true positive) to cost associated with a positive biomarker test in controls (false positive). Guidance is offered on soliciting the cost/benefit ratio. The calculations are based on the longstanding decision theory concept of providing a net benefit on average in the population, and they rely on some assumptions about uniformity of costs and benefits to those tested. RESULTS Calculations are illustrated with 3 applications: predicting colon cancer recurrence in stage 1 patients; predicting interval breast cancer (between mammography screenings); and screening for ovarian cancer. CONCLUSIONS It is feasible to specify target levels of biomarker performance that enable evaluation of the potential clinical impact of biomarkers in early-phase studies. Nevertheless, biomarkers meeting the criteria should still be tested rigorously in studies that measure the actual impact on patient outcomes of using the biomarker to make clinical decisions.

A breast pathology diagnosis provides the basis for clinical treatment and management decisions; however, its accuracy is inadequately understood.To quantify the magnitude of diagnostic disagreement among pathologists compared with a consensus panel reference diagnosis and to evaluate associated patient and pathologist characteristics.Study of pathologists who interpret breast biopsies in clinical practices in 8 US states.Participants independently interpreted slides between November 2011 and May 2014 from test sets of 60 breast biopsies (240 total cases, 1 slide per case), including 23 cases of invasive breast cancer, 73 ductal carcinoma in situ (DCIS), 72 with atypical hyperplasia (atypia), and 72 benign cases without atypia. Participants were blinded to the interpretations of other study pathologists and consensus panel members. Among the 3 consensus panel members, unanimous agreement of their independent diagnoses was 75%, and concordance with the consensus-derived reference diagnoses was 90.3%.The proportions of diagnoses overinterpreted and underinterpreted relative to the consensus-derived reference diagnoses were assessed.Sixty-five percent of invited, responding pathologists were eligible and consented to participate. Of these, 91% (N = 115) completed the study, providing 6900 individual case diagnoses. Compared with the consensus-derived reference diagnosis, the overall concordance rate of diagnostic interpretations of participating pathologists was 75.3% (95% CI, 73.4%-77.0%; 5194 of 6900 interpretations). Among invasive carcinoma cases (663 interpretations), 96% (95% CI, 94%-97%) were concordant, and 4% (95% CI, 3%-6%) were underinterpreted; among DCIS cases (2097 interpretations), 84% (95% CI, 82%-86%) were concordant, 3% (95% CI, 2%-4%) were overinterpreted, and 13% (95% CI, 12%-15%) were underinterpreted; among atypia cases (2070 interpretations), 48% (95% CI, 44%-52%) were concordant, 17% (95% CI, 15%-21%) were overinterpreted, and 35% (95% CI, 31%-39%) were underinterpreted; and among benign cases without atypia (2070 interpretations), 87% (95% CI, 85%-89%) were concordant and 13% (95% CI, 11%-15%) were overinterpreted. Disagreement with the reference diagnosis was statistically significantly higher among biopsies from women with higher (n = 122) vs lower (n = 118) breast density on prior mammograms (overall concordance rate, 73% [95% CI, 71%-75%] for higher vs 77% [95% CI, 75%-80%] for lower, P .001), and among pathologists who interpreted lower weekly case volumes (P .001) or worked in smaller practices (P = .034) or nonacademic settings (P = .007).In this study of pathologists, in which diagnostic interpretation was based on a single breast biopsy slide, overall agreement between the individual pathologists' interpretations and the expert consensus-derived reference diagnoses was 75.3%, with the highest level of concordance for invasive carcinoma and lower levels of concordance for DCIS and atypia. Further research is needed to understand the relationship of these findings with patient management.

Patients with Barrett's esophagus (BE) are at increased risk of developing esophageal adenocarcinoma (EAC). Clinical neoplastic progression risk factors, such as age and the length of the esophageal BE segment, have been identified. However, improved molecular biomarkers predicting increased progression risk are needed for improved risk assessment and stratification. Using real-time quantitative methylation-specific PCR, we screened 10 genes (HPP1, RUNX3, RIZ1, CRBP1, 3-OST-2, APC, TIMP3, p16, MGMT, p14) for promoter hypermethylation in 77 EAC, 93 BE, and 64 normal esophagus (NE) specimens. A subset of genes manifesting significant differences in methylation frequencies between BE and EAC was then analysed in 20 dysplastic specimens. All 10 genes except p14 were frequently methylated in EACs, with RUNX3, HPP1, CRBP1, RIZ1, and OST-2 representing novel methylation targets in EAC and/or BE. p16, RUNX3, and HPP1 displayed increasing methylation frequencies in BE vs EAC. Furthermore, these increases in methylation occurred early, at the interface between BE and low-grade dysplasia (LGD). To demonstrate the silencing effect of hypermethylation, we selected the EAC cells BIC1, in which the HPP1 promoter is natively methylated, and subjected them to 5-aza-2'-deoxycytidine (Aza-C) treatment. Real-time RT-PCR indicated increased HPP1 mRNA levels after 3 days of Aza-C treatment, as well as decreased levels of methylated HPP1 DNA. Hypermethylation of a subset of six genes (APC, TIMP3, CRBP1, p16, RUNX3, and HPP1) was then tested in a retrospective longitudinal study of 99 BE and nine LGD specimens obtained from 53 BE patients undergoing surveillance endoscopy. Only high-grade dysplasia (HGD) or EAC were defined as progression end points. Two patient groups were compared: eight progressors (P) and 45 nonprogressors (NP), using Cox proportional hazards models to determine the relative progression risks of age, BE segment length, and methylation events. Multivariate analyses revealed that only hypermethylation of p16 (odds ratio (OR) 1.74, 95% confidence interval (CI) 1.33-2.20), RUNX3 (OR 1.80, 95% CI 1.08-2.81), and HPP1 (OR 1.77, 95% CI 1.06-2.81) were independently associated with an increased risk of progression, whereas age, BE segment length, and hypermethylation of TIMP3, APC, or CRBP1 were not independent risk factors. In combined analyses, risk was detectable up to, but not earlier than, 2 years preceding neoplastic progression. Hypermethylation of p16, RUNX3, and HPP1 in BE or LGD may represent independent risk factors for the progression of BE to HGD or EAC. These findings have implications regarding risk stratification, early EAC detection, and the appropriate endoscopic surveillance interval for patients with BE.

There are clear clinical and public health needs to improve the early detection of breast cancer in order to save lives. Breast cancer is the leading cause of cancer death in women worldwide and is the second leading cause of cancer mortality in women in the United States. Although mammography is widely used to screen for breast cancer, it suffers both from false negative and false positive readings resulting in later diagnoses or unnecessary follow-up (e.g., biopsies), respectively. Improvement of breast cancer early detection may be possible through the measurement of minimally invasive, easy to perform blood based biomarkers. Our studies were performed on pre-clinical plasma samples collected up to 12.5 months before breast cancer diagnosis from cases and matched controls obtained from the Women’s Health Initiative (WHI) Observational Study (a prospective cohort of 93,676 post-menopausal women). We utilized high-density antibody microarray methods for both discovery and preliminarily validation on distinct sample sets for early detection biomarkers of estrogen receptor positive (ER+) breast cancer (around 70% of all breast cancers are classified as ER+). Antibody microarray features (i.e., spots) identified as having ≥30% sensitivity at 80% specificity were considered to have met the cutoff criteria and identified as potential markers. A total of 129 features, corresponding to 101 proteins, were identified as potential markers in one WHI dataset (90 cases vs. 90 controls). Of the 129 features (101 proteins), 47 (34 proteins) validated by our cutoffs and demonstrated over or under-expression in cases compared to controls consistent with the first dataset in the second set of WHI samples (121 cases vs. 121 controls). Given this level of reproducibility, we believe these biomarkers could be used to improve early detection of ER+ breast cancer for these clinical scenarios: 1. Informing timing of a subsequent mammogram in women with a negative screening mammogram; 2. Informing continuation of mammographic screening among women 75-79 years; 3. Prioritizing women who should be screened with mammography in areas with limited resources. We are currently performing further validation of the protein markers in Early Detection Research Network (EDRN) specimen reference set from patients at the time of diagnosis. We plan to test our markers on a pre-diagnostic cohort from the Cardiovascular Health Study using a different, more clinically applicable platform. We used the same high-density antibody microarray platform to measure autoantibody-autoantigen complexes and sialyl Lewis-A and –X modifications in one of the WHI sample sets and are now testing these markers in the EDRN set for validation. Given the strength of our biomarker candidates, the high quality sets of biospecimens we used, the study designs employed, and the clearly delineated clinical applications proposed, we believe these markers could have near-term clinical impact. Citation Format: Carly B. Garrison, Tracey Marsh, Matthew Buas, Yuzheng Zhang, Margaret Pepe, Paul D. Lampe, Christopher I. Li. Proteomic early detection biomarkers for ER+ breast cancer to improve mammography screening [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2017; 2017 Apr 1-5; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2017;77(13 Suppl):Abstract nr 713. doi:10.1158/1538-7445.AM2017-713

Little is known about the frequency of discordant diagnoses identified during research.To describe diagnostic discordance identified during research and apply a newly designed research framework for investigating discordance.Breast biopsy cases (N = 407) from registries in Vermont and New Hampshire were independently reviewed by a breast pathology expert. The following research framework was developed to assess those cases: (1) compare the expert review and study database diagnoses, (2) determine the clinical significance of diagnostic discordance, (3) identify and correct data errors and verify the existence of true diagnostic discrepancies, (4) consider the impact of borderline cases, and (5) determine the notification approach for verified disagreements.Initial overall discordance between the original diagnosis recorded in our research database and a breast pathology expert was 32.2% (131 of 407). This was reduced to less than 10% after following the 5-step research framework. Detailed review identified 12 cases (2.9%) with data errors (2 in the underlying pathology registry, 3 with incomplete slides sent for expert review, and 7 with data abstraction errors). After excluding the cases with data errors, 38 cases (9.6%) among the remaining 395 had clinically meaningful discordant diagnoses (κ = 0.82; SE, 0.04; 95% confidence interval, 0.76-0.87). Among these 38 cases, 20 (53%) were considered borderline between 2 diagnoses by either the original pathologist or the expert. We elected to notify the pathology registries and facilities regarding discordant diagnoses.Understanding the types and sources of diagnostic discordance uncovered in research studies may lead to improved scientific data and better patient care.

Pulmonary function is an important measure of disease severity and prognosis in cystic fibrosis (CF). It is generally expressed as a percentage of a predicted value, calculated using regression equations derived from a reference population. A number of reference equations are in widespread use. The purposes of this study were to determine: 1) the extent to which, for a given absolute FEV(1) value, percent of predicted (PPFEV(1)) values vary when derived by different reference equations; and 2) whether these differences affect conclusions of longitudinal and cross-sectional analyses. Subjects were all Caucasians 6-18 years old in the 1990 Cystic Fibrosis Foundation Registry. We found clinically important discrepancies in PPFEV(1) when calculated by the methods of Dockery et al. [Am Rev Respir Dis 1983;128:405-412] and Wang et al. [Pediatr Pulmonol 1993;15:75-78] as compared to Knudson et al. [Am Rev Respir Dis 1983;127:725-734] or Polgar and Promadhat [Pulmonary Function Testing in Children 1971; Philadelphia: W.B. Saunders]. In longitudinal analyses, the choice of reference equation resulted in varying apparent rates of decline in FEV(1). For example, among subjects ages 12-14 years in 1990, the decline in PPFEV(1) from 1990-1995 varied between 2-11%, depending on the choice of reference equation. In cross-sectional analyses, the choice of reference equation affected the distribution of subjects classified as having mild, moderate, or severe lung disease. CF physicians should be aware of the impact of choice of reference equation in both clinical care and research.

Early diagnosis of disease has potential to reduce morbidity and mortality. Biomarkers may be useful for detecting disease at early stages before it becomes clinically ap parent. Prostate-specific antigen (PSA) is one such marker for prostate cancer. This article is concerned with modeling receiver operating characteristic (ROC) curves as sociated with biomarkers at various times prior to the time at which the disease is detected clinically, by two methods. The first models the biomarkers statistically using mixed-effects regression models, and uses parameter estimates from these models to estimate the time-specific ROC curves. The second directly models the ROC curves as a function of time prior to diagnosis and may be implemented using software pack ages with binary regression or generalized linear model routines. The approaches are applied to data from 71 prostate cancer cases and 71 controls who participated in a lung cancer prevention trial. Two biomarkers for prostate cancer were considered: total serum PSA and the ratio of free to total PSA. Not surprisingly, both markers performed better as the interval between PSA measurement and clinical diagnosis decreased. Although the two markers performed similarly eight years prior to diagnosis, it appears that total PSA performed better than the ratio measure at times closer to diagnosis. The area under the ROC curve was consistently greater for total PSA than for the ratio four and two years prior to diagnosis and at the time of diagnosis. Key words: time- dependent ROC curves; biomarkers; diagnosis; prostate-specific antigen. (Med Decis Making 1999; 19:242-251)

Studies examining the association between an outcome variable and multiple predictors are common in medical research. Examples include epidemiologic studies of risk factors for disease and clinical studies of prognostic indicators for diseased subjects. This paper is concerned with the assessment of the associations between the outcome and each predictor separately, the so-called univariate associations. Comparisons between predictors in regards to the strengths of their association with the outcome are considered. We show that though such comparisons cannot be made with standard techniques, they can be made using an algorithm which performs all of the univariate analyses simultaneously. This is accomplished with a non-standard application of generalized estimating equation methods. Comparisons of univariate associations are shown to be the key analyses of interest in a retrospective longitudinal study of childhood predictors of adult obesity. We illustrate the methodology on data from this study.

Despite the heightened interest in developing biomarkers predicting treatment response that are used to optimize patient treatment decisions, there has been relatively little development of statistical methodology to evaluate these markers. There is currently no unified statistical framework for marker evaluation. This paper proposes a suite of descriptive and inferential methods designed to evaluate individual markers and to compare candidate markers. An R software package has been developed which implements these methods. Their utility is illustrated in the breast cancer treatment context, where candidate markers are evaluated for their ability to identify a subset of women who do not benefit from adjuvant chemotherapy and can therefore avoid its toxicity.

There is growing interest in markers that can be used to identify which patients are most likely to benefit from a treatment. For example, the Gail breast cancer risk prediction model may be useful for identifying a subset of older women for whom the benefit of tamoxifen for breast cancer prevention is likely to outweigh the harm. Two general classes of approaches to evaluating treatment selection markers have been developed. The first uses data on a cohort of untreated subjects to develop a risk prediction model, such as the Gail model, which is used to identify a high-risk subset of subjects. This model is paired with a measure of treatment effect to assess the impact of identifying and treating the high-risk subset. The second approach uses data from a randomized trial to model the treatment effect on a composite outcome that includes all effects of treatment (positive and negative). The treatment effect model is used to identify a subset of subjects with positive treatment effects and to assess the impact of identifying and treating this subset. We describe a framework that includes both existing approaches as special cases. In doing so, we review the existing approaches, clarify their underlying assumptions, and facilitate the evaluation of markers under less restrictive assumptions.

The effect of physician diagnostic variability on accuracy at a population level depends on the prevalence of diagnoses.To estimate how diagnostic variability affects accuracy from the perspective of a U.S. woman aged 50 to 59 years having a breast biopsy.Applied probability using Bayes' theorem.B-Path (Breast Pathology) Study comparing pathologists' interpretations of a single biopsy slide versus a reference consensus interpretation from 3 experts.115 practicing pathologists (6900 total interpretations from 240 distinct cases).A single representative slide from each of the 240 cases was used to estimate the proportion of biopsies with a diagnosis that would be verified if the same slide were interpreted by a reference group of 3 expert pathologists. Probabilities of confirmation (predictive values) were estimated using B-Path Study results and prevalence of biopsy diagnoses for women aged 50 to 59 years in the Breast Cancer Surveillance Consortium.Overall, if 1 representative slide were used per case, 92.3% (95% CI, 91.4% to 93.1%) of breast biopsy diagnoses would be verified by reference consensus diagnoses, with 4.6% (CI, 3.9% to 5.3%) overinterpreted and 3.2% (CI, 2.7% to 3.6%) underinterpreted. Verification of invasive breast cancer and benign without atypia diagnoses is highly probable; estimated predictive values were 97.7% (CI, 96.5% to 98.7%) and 97.1% (CI, 96.7% to 97.4%), respectively. Verification is less probable for atypia (53.6% overinterpreted and 8.6% underinterpreted) and ductal carcinoma in situ (DCIS) (18.5% overinterpreted and 11.8% underinterpreted).Estimates are based on a testing situation with 1 slide used per case and without access to second opinions. Population-adjusted estimates may differ for women from other age groups, unscreened women, or women in different practice settings.This analysis, based on interpretation of a single breast biopsy slide per case, predicts a low likelihood that a diagnosis of atypia or DCIS would be verified by a reference consensus diagnosis. This diagnostic grey zone should be considered in clinical management decisions in patients with these diagnoses.National Cancer Institute.

Optimal triage of patients at risk for critical illness requires accurate risk prediction, yet few data on the performance criteria required of a potential biomarker to be clinically useful exists.We studied an adult cohort of nonarrest, nontrauma emergency medical services encounters transported to a hospital from 2002 to 2006. We simulated hypothetical biomarkers increasingly associated with critical illness during hospitalization and determined the biomarker strength and sample size necessary to improve risk classification beyond a best clinical model.Of 57,647 encounters, 3121 (5.4%) were hospitalized with critical illness and 54,526 (94.6%) without critical illness. The addition of a moderate-strength biomarker (odds ratio, 3.0, for critical illness) to a clinical model improved discrimination (c statistic, 0.85 vs 0.8; P.01) and reclassification (net reclassification improvement, 0.15; 95% confidence interval, 0.13-0.18) and increased the proportion of cases in the highest-risk category by +8.6% (95% confidence interval, 7.5%-10.8%). Introducing correlation between the biomarker and physiological variables in the clinical risk score did not modify the results. Statistically significant changes in net reclassification required a sample size of at least 1000 subjects.Clinical models for triage of critical illness could be significantly improved by incorporating biomarkers, yet substantial sample sizes and biomarker strength may be required.

One hundred forty-seven patients with hematologic diseases and treated by allogeneic marrow transplants received graft-versus-host disease (GVHD) prevention with methotrexate and cyclosporine. In addition, 73 of the 147 patients were randomized to receive methylprednisolone during the first 35 days after transplant to improve GVHD prevention, whereas 74 patients were randomized not to receive methylprednisolone. The randomized trial enabled us to examine whether methylprednisolone increased the risk of infection after marrow grafting. Charts of study patients were analyzed retrospectively for infection events including bacteremia, septicemia, and fungemia. The randomization was stratified by diagnosis, patient age, genotypic HLA identity, and assignment to laminar airflow room isolation. All patients were given a short course of methotrexate (no longer than 11 days) and cyclosporine for no longer than 180 days after marrow transplantation. Methylprednisolone was begun on the day of marrow grafting at a dose of 1 mg/kg body weight intravenously in divided AM and PM doses through day 22. Methylprednisolone was administered at a dose of 0.5 mg/kg in divided doses from days 22 through 35, and then discontinued. Infections were analyzed for the time interval ending on day 65 after transplantation, which included the period of methylprednisolone administration and 1 month thereafter. Seventy-one episodes of first infection events were observed in patients receiving methylprednisolone compared with 47 episodes in patients not receiving the drug. Predominant infections were bacteremias, followed in descending order by fungemias and septicemias. The most prevalent organisms cultured were gram-positive bacteria, especially coagulase-negative Staphylococcus and Streptococcus species. Pseudomonas species were the most common gram negative bacteria, and the most prevalent fungus was Candida albicans. Multivariable Cox regression analysis showed that patients receiving methylprednisolone had a 1.5 times higher risk of infection (P = .03), with acute GVHD being another independent risk factor for infections (P = .005). Methylprednisolone, when added to GVHD prevention by methotrexate and cyclosporine, increases the risk of infection during the early posttransplantation period.

Treatment selection markers, sometimes called predictive markers, are factors that help clinicians select therapies that maximize good outcomes and minimize adverse outcomes for patients. Existing statistical methods for evaluating a treatment selection marker include assessing its prognostic value, evaluating treatment effects in patients with a restricted range of marker values, and testing for a statistical interaction between marker value and treatment. These methods are inadequate, because they give misleading measures of performance that do not answer key clinical questions about how the marker might help patients choose treatment, how treatment decisions should be made on the basis of a continuous marker measurement, what effect using the marker to select treatment would have on the population, or what proportion of patients would have treatment changes on the basis of marker measurement. Marker-by-treatment predictiveness curves are proposed as a more useful aid to answering these clinically relevant questions, because they illustrate treatment effects as a function of marker value, outcomes when using or not using the marker to select treatment, and the proportion of patients for whom treatment recommendations change after marker measurement. Randomized therapeutic clinical trials, in which entry criteria and treatment regimens are not restricted by the marker, are also proposed as the basis for constructing the curves and evaluating and comparing markers.

One hundred twelve patients less than 36 years old received marrow grafts from unrelated donors as treatment for hematologic malignancy. Seventy donor/recipient pairs were phenotypically identical for HLA-A, - B, and -D, while 42 had a “minor” disparity at one HLA locus. There was an increase in the risk of acute graft-versus-host disease (GVHD) in patients receiving HLA-partially matched grafts compared with those receiving HLA-matched grafts (51% v 36% probability of grades III-IV acute GVHD). However, in this cohort of patients, there was no significant difference in survival (at 1.5 years, 46% v 51% for good- risk patients, 44% v 30% for poor-risk patients). This finding suggests that some degree of HLA disparity can be tolerated in young patients transplanted from unrelated donors for malignant disease, thus making transplantation an option available to larger numbers of patients.

Previous studies of risk factors for acute graft-versus-host disease (GVHD) involved patients receiving predominantly single-agent prophylaxis. Therefore, a retrospective analysis was performed on 446 patients, from a single institution, who received transplants of marrow from HLA-identical siblings and the combination of cyclosporine (CSP) and methotrexate (MTX) to determine risk factors for acute GVHD associated with this more effective form of GVHD prophylaxis. The incidences of Grades II-IV and Grades III-IV (severe) acute GVHD were 35% and 16%, respectively. Increased clinical grades of acute GVHD in patients without advanced malignant disease were associated with a decreased survival. In a multivariate Cox regression analysis, risk factors associated with the onset of Grades II-IV acute GVHD were sex mismatch and donor parity (P = .001), increased dose of total body irradiation (TBI) (P = .001), and reduction to less than 80% of the scheduled dose of MTX (P = .02) or CSP (P = .02). The multivariate analysis indicated a relative risk of 1.37 for acute GVHD in a group defined as having advanced malignant disease at transplant; however, this difference failed to reach conventional levels of statistical significance (P = .07). Reduction of MTX and CSP occurred in up to 36% and 44% of patients, respectively, primarily because of renal or hepatic dysfunction. The periods of increased risk for the onset of acute GVHD were up to 1 week after a reduction of MTX and 2 weeks after a reduction in CSP. When only patients who developed Grades II-IV acute GVHD were considered, the more severe acute GVHD of Grades III-IV was associated with increased patient age of 40 years or greater (P = .05) and dose reductions of CSP (P = .008). Serologic status of patient and donor for cytomegalovirus (CMV), HLA antigens in the A and B loci, and isolation in a laminar air flow room during marrow transplantation, all previously identified as risk factors for acute GVHD, were not confirmed as risk factors in this study population. The toxicity of MTX and CSP and the development of acute GVHD from inadequate immunosuppression because of dose reduction warrants further trials with potentially less toxic immunosuppressive agents. Risk factors for acute GVHD should be considered in clinical management and in the design of clinical trials.

Sixty-eight patients with moderate (n = 15) or severe (n = 53) aplastic anemia were entered into a prospective, randomized, two-arm treatment study comparing antihuman thymocyte globulin (ATG), lower-dose methylprednisolone (LDM) and oxymetholone to ATG, higher-dose methylprednisolone (HDM), and oxymetholone. There were no differences between the two groups when comparing age, sex, etiology of aplasia, disease duration, severity of aplasia, or pretherapy granulocyte counts. Side effects of LDM and HDM were similar. Of the 64 patients evaluable for response to therapy, 12 of 33 (36%) who received LDM had complete, partial, or minimal responses compared with 15 of 31 patients (48%) who received HDM (P = .33). Actuarial survival at 4 years is 43% for patients in the LDM group and 47% for patients in the HDM group (P = .99). Causes of death included hemorrhage, infection, evolution to acute leukemia, and complications of subsequent bone marrow transplantation. Long-term complications included paroxysmal nocturnal hemoglobinuria (n = 3), evolution to myelodysplasia or acute leukemia (n = 6), and recurrent aplasia (n = 6). We were unable to show a significant difference in toxicity, response rate, or survival for patients treated with ATG, oxymetholone, and LDM compared with patients who received ATG, oxymetholone, and HDM.

Summary. Nineteen patients with leukaemia, preleukaemia, and aplastic anaemia were treated by marrow transplantation from HLA-identical siblings. All were given postgrafting immunosuppression with a combination of methotrexate (days 1,3,6 and 11) and cyclosporin (days -1 to 180). In an attempt at reducing cyclosporin-associated toxicity, we explored whether the cyclosporin dose during the first 2 weeks could be decreased by 50% (from 3.0 to 1.5 mg/kg/d intravenously) without adversely affecting the incidence, onset, and severity of acute graft-versus-host disease (GVHD) and overall survival. Results from this pilot study were compared to those of a matched cohort of 38 patients given a standard dose of 3.0 mg cyclosporin/kg/d starting on day -1. The cumulative incidence of grade II and III acute GVHD in the ‘low dose’ cyclosporin group was 42% compared to 51% in the ‘standard dose’ group (P= 0.60). Three-year survival was 63% and 54% respectively (P=0.59). Patients receiving the reduced cyclosporin dose during the first 14 d appeared to have less hepatotoxicity, and the methotrexate and cyclosporin doses administered were closer to the doses intended per protocol. We suggest that‘low dose’cyclosporin from day -1 to day 15 postgrafting might be as effective as ‘standard dose’ cyclosporin during that time period for the prevention of acute GVHD.

The activation signals leading to proliferation of normal and leukemic CD3+ large granular lymphocytes (LGL) were studied in vitro. Anti-CD3 monoclonal antibody (MoAb) alone (P less than .01) and recombinant interleukin-2 (IL-2) alone (P less than .01) caused significant stimulation of peripheral blood mononuclear cells (PBMC) from four CD3+ LGL leukemia patients, as measured in a 3H-thymidine incorporation assay. Recombinant interleukin-4 (IL-4) alone had no effect (P = .11). The combination signals of anti-CD3 MoAb and either IL-2 or IL-4 produced a proliferative response greater than anti-CD3 MoAb alone (P less than .01) or lymphokine alone (P less than .01). Leukemic LGL, purified by two-color sorting, were subsequently activated by anti-CD3 MoAb and IL-2 and assessed for DNA content by viable Hoechst No. 33342 (HO) staining. Results of these studies demonstrated that leukemic LGL were stimulated directly by anti-CD3 MoAb and IL-2, with the percentage of cells in cell cycle (S + G2/M) ranging from 16% to 72%. Normal CD3+ LGL were also stimulated to enter the cell cycle by anti-CD3 and IL-2. These results show that leukemic LGL proliferate in vitro after activation through the T-cell receptor and/or lymphokine.

We propose an estimator of the prevalence of a transient condition among surviving patients using right censored data. The prevalence of opportunistic infection among surviving AIDS patients and the probability of being in tumour response following cancer therapy conditional on being alive are two examples of such functions. In essence these functions describe a major aspect of the quality of life for surviving patients and may be useful when viewed in conjunction with the survival curves themselves. The method is illustrated using data from a randomized trial of bone marrow transplant patients where the prevalence of chronic graft-versus-host disease is of interest. The non-parametric estimator which we have proposed is contrasted with estimators derived from Markov and semi-Markov models.

Less than 40% of the patients who could benefit from marrow transplantation have an HLA-matched relative who can serve as a donor. For this reason, several centers have explored marrow transplantation from other categories of donors. This retrospective study analyzes the results of marrow transplantation for 52 patients receiving grafts from HLA-A,B,DR,Dw-phenotypically matched, MLC-compatible, unrelated volunteer donors compared to a disease, disease-stage, and age-matched cohort of 104 patients transplanted from HLA-genotypically identical sibling donors. The patients transplanted from unrelated donors had an increased incidence of grade II-IV acute graft-versus-host disease compared to patients transplanted from related donors (79% vs. 36%, P much less than 0.001). However, the probability of relapse-free survival appears similar in the two groups (P = 0.39 over all, with estimates of 41% vs. 46% at 1 year). We conclude from this preliminary data that marrow transplantation from HLA-matched unrelated donors should be considered in most, if not all, circumstances where transplantation from an HLA-matched sibling would be indicated if such a donor were available.

The recent epidemiologic and clinical literature is filled with studies evaluating statistical models for predicting disease or some other adverse event. Risk stratification tables are a new way to evaluate the benefit of adding a new risk marker to a risk prediction model that includes an established set of markers. This approach involves cross-tabulating risk predictions from models with and without the new marker. In this article, the authors use examples to show how risk stratification tables can be used to compare 3 important measures of model performance between the models with and those without the new marker: the extent to which the risks calculated from the models reflect the actual fraction of persons in the population with events (calibration); the proportions in which the population is stratified into clinically relevant risk categories (stratification capacity); and the extent to which participants with events are assigned to high-risk categories and those without events are assigned to low-risk categories (classification accuracy). They detail common misinterpretations and misuses of the risk stratification method and conclude that the information that can be extracted from risk stratification tables is an enormous improvement over commonly reported measures of risk prediction model performance (for example, c-statistics and Hosmer-Lemeshow tests) because it describes the value of the models for guiding medical decisions.

The concept of covariate adjustment is well established in therapeutic and etiologic studies. However, it has received little attention in the growing area of medical research devoted to the development of markers for disease diagnosis, screening, or prognosis, where classification accuracy, rather than association, is of primary interest. In this paper, the authors demonstrate the need for covariate adjustment in studies of classification accuracy, discuss methods for adjusting for covariates, and distinguish covariate adjustment from several other related, but fundamentally different, uses for covariates. They draw analogies and contrasts throughout with studies of association.

One hundred forty-seven consecutive patients with leukemia, myelodysplastic syndrome, or aplastic anemia were treated by marrow grafts from genotypically HLA-identical siblings (n = 122) or HLA- haploidentical family members (n = 25). Haploidentical recipients differed from their donors for no more than one HLA locus on the nonshared haplotype. All were given postgrafting immunosuppression with a combination of methotrexate and cyclosporine. In a randomized study we explored whether prednisone administered from day 0 through 35 along with methotrexate/cyclosporine could improve prevention of acute graft- versus-host disease (GVHD). The GVHD incidence in patients not given prednisone was comparable with that previously reported with methotrexate/cyclosporine. Unexpectedly, significant increases in acute and also chronic GVHD were seen in HLA-identical recipients administered prednisone, but not in the small number of patients administered HLA-nonidentical grafts. However, the resultant increase in transplant-related mortality in patients administered prednisone was offset by an increase in leukemic relapse in patients not administered prednisone, presumably related to the absence of a graft-versus- leukemia effect. Therefore, overall disease-free survival of the two groups of patients was comparable, with slightly more than 50% of the patients being alive at more than 2 years after transplantation. We speculated that prednisone adversely affected GVHD prophylaxis, interfering with methotrexate's cell cycle-dependent suppression of donor lymphocyte proliferation in response to host antigens. In a pilot study we explored whether beginning prednisone on day 15, after completion of methotrexate administration, would avoid this adverse effect. The GVHD incidence in patients administered methotrexate/cyclosporine along with “late” prednisone was comparable with that in patients not administered prednisone. We conclude that methotrexate/cyclosporine is effective in decreasing the incidence of grade II through IV GVHD, and that the addition of prednisone to this regimen is not beneficial in recipients of HLA-identical marrow grafts.

An ELISA was used to study long-term immunity and immunization responses to tetanus toxoid in 48 bone marrow transplant recipients. Among patients who were seropositive to tetanus before transplant, 51% had lost their seropositivity 1 year later. All patients who were not reimmunized with tetanus toxoid were seronegative 2 years after transplant. All patients who were seronegative before transplant remained seronegative 1 year later regardless of the donor's serologic status. There was no difference in the ability to remain seropositive to tetanus toxoid between patients with and without chronic graft-versus-host disease. Of 21 patients immunized with one dose of tetanus toxoid 1 year after transplant, 14 were seronegative at the time of immunization (response rate, 64%). At 1 year after immunization, 7 remained seropositive. Ten patients were reimmunized with two doses of tetanus toxoid. All responded and 90% remained seropositive 1 year later. When 21 patients were primarily immunized with three doses of tetanus toxoid, all patients seronegative at immunization responded and all tested patients remained seropositive 2 years later. The immunization responses were significantly superior in patients receiving three doses compared with those who received one. Reimmunization with tetanus toxoid of long-term survivors after marrow transplant seems necessary. A three-dose immunization schedule is recommended to obtain an adequate immune response.

Developing biomarkers that can predict whether patients are likely to benefit from an intervention is a pressing objective in many areas of medicine. Recent guidance documents have recommended that the accuracy of predictive biomarkers, ie, sensitivity, specificity, and positive and negative predictive values, should be assessed. We clarify the meanings of these entities for predictive markers and demonstrate that generally they cannot be estimated from data without making strong untestable assumptions. Language suggesting that predictive biomarkers can identify patients who benefit from an intervention is also widespread. We show that in general one cannot estimate the chance that a patient will benefit from treatment. We recommend instead that predictive biomarkers be evaluated with respect to their ability to predict clinical outcomes among patients treated and among patients receiving standard of care, and the population impact of treatment rules based on those predictions. Ideally these entities are estimated from a randomized trial comparing the experimental intervention with standard of care.

Background: Triple negative breast cancers (TNBC) comprise 15-20% of all breast cancers and frequently present as interval cancers with high proliferative rates and increased risk of mortality. There is a clinical need for biomarkers for the early detection of TNBC to complement radiologic imaging. No plasma biomarkers for TNBC currently exist. The purpose of this study is to evaluate a panel of novel plasma biomarkers for TNBC for the discrimination of TNBC and benign breast disease as a crucial step in identifying a panel of plasma biomarkers for early detection. Methods: In a multicenter collaboration between the NCI EDRN and the CPTAC consortium, we conducted a prospective blinded phase II biomarker study that evaluated 76 candidate TNBC plasma biomarkers. Plasma samples collected at the time of diagnosis from 65 TNBC cases and 195 matched controls with benign breast disease without atypia were identified from multiple clinical sites. The samples were distributed as blinded aliquots to the biomarker laboratories for protein and autoantibody detection. Candidate protein (n=54) and autoantibody (n=22) biomarkers were selected and ranked prior to evaluation. All results were centrally analyzed. The sensitivity at 95% specificity was calculated for each biomarker. Effects of age, race, and specimen source on biomarkers were evaluated. Logistic regression was used to assess complementarity of biomarkers. The top three biomarkers underwent verification with an independent set of 60 TNBC cases and 180 matched controls from women undergoing mammography. Results: Statistically significant differences in case versus control signals were observed for 3 biomarkers with sensitivities of 17-23% at 95% specificity (p Conclusion: We have developed a pipeline strategy for the validation of plasma biomarkers for detection of breast cancer. At least three biomarkers for TNBC were confirmed in this study. Further evaluation of these biomarkers for early detection is ongoing. Citation Format: Karen S Anderson, Margaret Pepe, Jeffrey Marks, Paul Engstrom, Christos Patriotis, Richard Zangar, Steven Skates, Paul Lampe, Joshua LaBaer, Christopher I Li. A blinded multicenter phase II study of a panel of plasma biomarkers for the detection of triple negative breast cancer [abstract]. In: Proceedings of the Thirty-Seventh Annual CTRC-AACR San Antonio Breast Cancer Symposium: 2014 Dec 9-13; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2015;75(9 Suppl):Abstract nr P4-02-05.

In this issue, Leening and colleagues discuss limitations and controversies surrounding the NRI and propose a systematic approach for presenting NRI analysis. The editorialists recommend that resea...

Plasma ghrelin levels rise before meals and fall rapidly afterward. If ghrelin is a physiological meal-initiation signal, then a large oral caloric load should suppress ghrelin levels more than a small caloric load, and the request for a subsequent meal should be predicted by recovery of the plasma ghrelin level. To test this hypothesis, 10 volunteers were given, at three separate sessions, liquid meals (preloads) with widely varied caloric content (7.5%, 16%, or 33% of total daily energy expenditure) but equivalent volume. Preloads were consumed at 0900 h, and blood was sampled every 20 min from 0800 h until 80 min after subjects spontaneously requested a meal. The mean (+/- SE) intervals between ingestion of the 7.5%, 16%, and 33% preloads and the subsequent voluntary meal requests were 247 +/- 24, 286 +/- 20, and 321 +/- 27 min, respectively (P = 0.015), and the nadir plasma ghrelin levels were 80.2 +/- 2.8%, 72.7 +/- 2.7%, and 60.8 +/- 2.7% of baseline (the 0900 h value), respectively (P0.001). A Cox regression analysis failed to show a relationship between ghrelin profile and the spontaneous meal request. We conclude that the depth of postprandial ghrelin suppression is proportional to ingested caloric load but that recovery of plasma ghrelin is not a critical determinant of intermeal interval.

SUMMARY In early detection of disease, combinations of biomarkers promise improved discrimination over diagnostic tests based on single markers. An example of this is in prostate cancer screening, where additional markers have been sought to improve the specificity of the conventional Prostate-Specific Antigen (PSA) test. A marker of particular interest is the percent free PSA. Studies evaluating the benefits of percent free PSA reflect the need for a methodological approach that is statistically valid and useful in the clinical setting. This article presents methods that address this need. We focus on and-or combinations of biomarker results that we call logic rules and present novel definitions for the ROC curve and the area under the curve (AUC) that are applicable to this class of combination tests. Our estimates of the ROC and AUC are amenable to statistical inference including comparisons of tests and regression analysis. The methods are applied to data on free and total PSA levels among prostate cancer cases and matched controls enrolled in the Physicians’ Health Study.