Search

Prediction models aim to use available data to predict a health state or outcome that has not yet been observed. Prediction is primarily relevant to clinical practice, but is also used in research, and administration. While prediction modeling involves estimating the relationship between patient factors and outcomes, it is distinct from casual inference. Prediction modeling thus requires unique considerations for development, validation, and updating. This document represents an effort from editors at 31 respiratory, sleep, and critical care medicine journals to consolidate contemporary best practices and recommendations related to prediction study design, conduct, and reporting. Herein, we address issues commonly encountered in submissions to our various journals. Key topics include considerations for selecting predictor variables, operationalizing variables, dealing with missing data, the importance of appropriate validation, model performance measures and their interpretation, and good reporting practices. Supplemental discussion covers emerging topics such as model fairness, competing risks, pitfalls of "modifiable risk factors", measurement error, and risk for bias. This guidance is not meant to be overly prescriptive; we acknowledge that every study is different, and no set of rules will fit all cases. Additional best practices can be found in the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines, to which we refer readers for further details., SCOPUS: ar.j, info:eu-repo/semantics/published

OBJECTIVES Acute renal replacement therapy in patients with sepsis has increased dramatically with substantial costs. However, the extent of variability in use across hospitals-and whether greater use is associated with better outcomes-is unknown. DESIGN Retrospective cohort study. SETTING Nationwide Inpatient Sample in 2011. PATIENTS Eighteen years old and older with sepsis and acute kidney injury admitted to hospitals sampled by the Nationwide Inpatient Sample in 2011. INTERVENTIONS We estimated the risk- and reliability-adjusted rate of acute renal replacement therapy use for patients with sepsis and acute kidney injury at each hospital. We examined the association between hospital-specific renal replacement therapy rate and in-hospital mortality and hospital costs after adjusting for patient and hospital characteristics. MEASUREMENTS AND MAIN RESULTS We identified 293,899 hospitalizations with sepsis and acute kidney injury at 440 hospitals, of which 6.4% (n = 18,885) received renal replacement therapy. After risk and reliability adjustment, the median hospital renal replacement therapy rate for patients with sepsis and acute kidney injury was 3.6% (interquartile range, 2.9-4.5%). However, hospitals in the top quintile of renal replacement therapy use had rates ranging from 4.8% to 13.4%. There was no significant association between hospital-specific renal replacement therapy rate and in-hospital mortality (odds ratio per 1% increase in renal replacement therapy rate: 1.03; 95% CI, 0.99-1.07; p = 0.10). Hospital costs were significantly higher with increasing renal replacement therapy rates (absolute cost increase per 1% increase in renal replacement therapy rate: $1,316; 95% CI, $157-$2,475; p = 0.03). CONCLUSIONS Use of renal replacement therapy in sepsis varied widely among nationally sampled hospitals without associated differences in mortality. Improving renal replacement standards for the initiation of therapy for sepsis may reduce healthcare costs without increasing mortality.

Admission to an intensive care unit (ICU) may be beneficial to patients with pneumonia with uncertain ICU needs; however, evidence regarding the association between ICU admission and mortality for other common conditions is largely unknown.To estimate the relationship between ICU admission and outcomes for hospitalized patients with exacerbation of chronic obstructive pulmonary disease (COPD), exacerbation of heart failure (HF), or acute myocardial infarction (AMI).We performed a retrospective cohort study of all acute care hospitalizations from 2010 to 2012 for U.S. fee-for-service Medicare beneficiaries aged 65 years and older admitted with COPD exacerbation, HF exacerbation, or AMI. We used multivariable adjustment and instrumental variable analysis to assess each condition separately. The instrumental variable analysis used differential distance to a high ICU use hospital (defined separately for each condition) as an instrument for ICU admission to examine marginal patients whose likelihood of ICU admission depended on the hospital to which they were admitted. The primary outcome was 30-day mortality. Secondary outcomes included hospital costs.Among 1,555,798 Medicare beneficiaries with COPD exacerbation, HF exacerbation, or AMI, 486,272 (31%) were admitted to an ICU. The instrumental variable analysis found that ICU admission was not associated with significant differences in 30-day mortality for any condition. ICU admission was associated with significantly greater hospital costs for HF ($11,793 vs. $9,185, P 0.001; absolute increase, $2,608 [95% confidence interval, $1,377-$3,840]) and AMI ($19,513 vs. $14,590, P 0.001; absolute increase, $4,922 [95% confidence interval, $2,665-$7,180]), but not for COPD.ICU admission did not confer a survival benefit for patients with uncertain ICU needs hospitalized with COPD exacerbation, HF exacerbation, or AMI. These findings suggest that the ICU may be overused for some patients with these conditions. Identifying patients most likely to benefit from ICU admission may improve health care efficiency while reducing costs.

In the United States, approximately 20% of patients hospitalized with pneumonia are readmitted to a hospital within 30 days. Given the significant costs and healthcare system use resulting from unplanned readmissions, pneumonia readmission rates are a target of national quality measures. Patient do-not-resuscitate (DNR) status strongly influences hospital pneumonia mortality measures; however, associations between DNR status and 30-day readmissions after pneumonia are unclear.Determine the effect of accounting for patient DNR status on hospital readmission measures for pneumonia.After excluding patients with missing data, those who died during the index hospitalization, those who were discharged against medical advice, those who did not reside in California, and those admitted to low pneumonia case-volume hospitals, we identified 30-day unplanned readmissions after an index pneumonia hospitalization from the 2011 California State Inpatient Database. We used hierarchical logistic regression to determine the association between early DNR status (within 24 hours of admission) and 30-day readmission and hospital risk-adjusted readmission rates.We identified 68,691 hospitalizations for pneumonia across 321 hospitals. Patients with early DNR orders were less likely to be readmitted within 30 days (20.0% vs. 22.5%, adjusted odds ratio [aOR], 0.93; 95% confidence interval [CI], 0.88-0.99). Patients with pneumonia admitted to high-versus-low DNR rate hospitals were at lower risk for readmission (DNR rate quartile 4 vs. quartile 1, aOR, 0.62; 95% CI, 0.55-0.70), regardless of individual DNR status. Higher hospital risk-adjusted DNR rates were strongly associated with lower risk-adjusted readmission rates (r = -0.44; P 0.0001). Inclusion of early DNR status in risk-adjusted readmission models changed ranking categories for 7/321 (2.2%) hospitals, with 2 hospitals no longer labeled as "under-performing outliers."Patients with an early DNR order have a lower risk for readmission after a pneumonia hospitalization. Unmeasured DNR status weakly confounds hospital readmission measures; accounting for patient DNR status would alter readmission ratings for a small number of hospitals.

Identifying patients with acute respiratory distress syndrome (ARDS) is a recognized challenge. Experts often have only moderate agreement when applying the clinical definition of ARDS to patients. However, no study has fully examined the implications of low reliability measurement of ARDS on clinical studies.To investigate how the degree of variability in ARDS measurement commonly reported in clinical studies affects study power, the accuracy of treatment effect estimates, and the measured strength of risk factor associations.We examined the effect of ARDS measurement error in randomized clinical trials (RCTs) of ARDS-specific treatments and cohort studies using simulations. We varied the reliability of ARDS diagnosis, quantified as the interobserver reliability (κ-statistic) between two reviewers. In RCT simulations, patients identified as having ARDS were enrolled, and when measurement error was present, patients without ARDS could be enrolled. In cohort studies, risk factors as potential predictors were analyzed using reviewer-identified ARDS as the outcome variable.Lower reliability measurement of ARDS during patient enrollment in RCTs seriously degraded study power. Holding effect size constant, the sample size necessary to attain adequate statistical power increased by more than 50% as reliability declined, although the result was sensitive to ARDS prevalence. In a 1,400-patient clinical trial, the sample size necessary to maintain similar statistical power increased to over 1,900 when reliability declined from perfect to substantial (κ = 0.72). Lower reliability measurement diminished the apparent effectiveness of an ARDS-specific treatment from a 15.2% (95% confidence interval, 9.4-20.9%) absolute risk reduction in mortality to 10.9% (95% confidence interval, 4.7-16.2%) when reliability declined to moderate (κ = 0.51). In cohort studies, the effect on risk factor associations was similar.ARDS measurement error can seriously degrade statistical power and effect size estimates of clinical studies. The reliability of ARDS measurement warrants careful attention in future ARDS clinical studies.

Published studies suggest hospital costs on Day 1 in the intensive care unit (ICU) far exceed those of subsequent days, when costs are relatively stable. Yet, no study stratified patients by ICU type.To determine whether daily cost patterns differ by ICU type.We performed a retrospective study of adults admitted to five ICUs (two surgical: quaternary surgical ICU [SICU quat] and quaternary cardiac surgical ICU [CSICU quat]; two medical: tertiary medical ICU [MICU tertiary] and quaternary medical ICU [MICU quat]; one general: community medical surgical ICU [MSICU comm]) at Montefiore Medical Center in the Bronx, New York during 2013. After excluding costs clearly accrued outside the ICU, daily hospital costs were merged with clinical data. Patterns of daily unadjusted costs were evaluated in each ICU using median regression. Generalized estimating equations with first-order autocorrelation were used to identify factors independently associated with daily costs.Unadjusted daily costs were higher on Day 1 than on subsequent days only for surgical ICUs-SICU quat (median [interquartile range], $2,636 [$1,834-$4,282] on Day 1 vs. $1,840 [$1,501-$2,332] on Day 2; P0.001) and CSICU quat ($5,166 [$3,136-$9,493] on Day 1 vs. $2,060 [$1,336-$2,528] on Day 2; P0.001). In nonsurgical ICUs, there was no change (MICU tertiary P = 0.12) or a small increase (MSICU comm P = 0.03; MICU quat P = 0.01) in cost from Days 1 to 2. After multivariate adjustment, there remained a significant decrease in cost from ICU Day 1 to 2 in surgical units with statistically similar Day 1 and 2 costs for other ICUs.Higher Day 1 costs are not seen in patients admitted to medical/nonsurgical ICUs.

Purpose The purpose of the study is to describe the role of human metapneumovirus (hMPV) infection in critical illness and acute respiratory distress syndrome (ARDS). Materials and methods We collected clinical and demographic information from a retrospective chart review, comparing patients with and without an intensive care unit (ICU) admission. Among patients admitted to the ICU, we assessed whether hMPV was “unlikely,” “possibly,” or “likely” the reason for ICU admission, based on a prespecified definition, and whether the patient met criteria for ARDS. Results We identified 128 hospitalized adults with hMPV infection. Forty hospitalized patients (31%) with hMPV infection required admission to the ICU. Among patients cared for in the ICU, hMPV was “possibly” the reason for ICU admission in 55% of patients and “likely” the reason in 38%. Forty-eight percent of ICU patients met criteria for ARDS. Although most patients admitted to the ICU had significant comorbidities or were immunosuppressed, 6 patients requiring ICU admission had more minor comorbidities and no underlying immunosuppression. Conclusions Although most patients hospitalized with hMPV had chronic cardiac or pulmonary disease, hMPV can also be associated serious respiratory illness and ARDS in adult patients without significant comorbidities or immunosuppression.

Background The treatment for patients with acute myocardial infarction (AMI) was transformed by the introduction of intensive care units (ICUs), yet we know little about how contemporary hospitals use this resource-intensive setting and whether higher use is associated with better outcomes. Methods We identified 114,136 adult hospitalizations for AMI from 307 hospitals in the 2009 to 2010 Premier database using codes from the International Classification of Diseases, Ninth Revision, Clinical Modification . Hospitals were stratified into quartiles by rates of ICU admission for AMI patients. Across quartiles, we examined in-hospital risk-standardized mortality rates and usage rates of critical care therapies for these patients. Results Rates of ICU admission for AMI patients varied markedly among hospitals (median 48%, Q1-Q4 20%-71%, range 0%-98%), and there was no association with in-hospital risk-standardized mortality rates (6% all quartiles, P = .7). However, hospitals admitting more AMI patients to the ICU were more likely to use critical care therapies overall (mechanical ventilation [from Q1 with lowest rate of ICU use to Q4 with highest rate 13%-16%], vasopressors/inotropes [17%-21%], intra-aortic balloon pumps [4%-7%], and pulmonary artery catheters [4%-5%]; P for trend Conclusions Rates of ICU admission for patients with AMI vary substantially across hospitals and were not associated with differences in mortality, but were associated with greater use of critical care therapies. These findings suggest uncertainty about the appropriate use of this resource-intensive setting and a need to optimize ICU triage for patients who will truly benefit.

The U.S. healthcare system often fails to deliver safe, effective, high-value, and patient-centered care (1). In an effort to improve the quality of healthcare, policymakers and payers have rapidly expanded their use of performance measurement programs during the last 2 decades. Examples include regional and national pay for performance and value-based purchasing programs, as well as public reporting programs such as Medicare’s Hospital Compare. A key step in ensuring such programs are improving the quality of care is rigorous program evaluation. By examining a program’s effectiveness, researchers can assure all stakeholders involved that an ongoing investment in the program is worthwhile. This work also informs the planning and implementation of future programs. To date, the vast majority of performance measurement programs in the United States and their subsequent evaluations have focused on outpatient or general inpatient medical and surgical care. Very few studies have examined the effect of performance measurement on care specific to the intensive care unit (ICU). The California Hospital Assessment and Reporting Taskforce (CHART) program collected and reported quality measures for California hospitals on the website CalHospitalCompare.org between 2007 and 2011 (2). A unique feature of the CHART program was the public reporting of risk-adjusted ICU mortality rates. The original goal of CHART was to make hospital performance data publicly available so high-quality hospitals would be rewarded by gaining market share and receive higher reimbursement rates for the care they provide. However, the California Hospital Association withdrew support of the initiative in 2011, citing the substantial resources required for data collection and the failure of health insurance plans to reward high-performing hospitals (3). The association also felt the newly created national public reporting program, HospitalCompare.gov, by the Center for Medicare & Medicaid Services (CMS), could provide an appropriate alternative source of quality measurement data, despite the absence of a requirement to include ICU mortality rates in the national program. No study evaluating the effectiveness of publically reporting ICU mortality rates was available to inform the decision to discontinue the program. In this month’s issue of AnnalsATS, Reineck and colleagues (pp. 57–63) perform an evaluation of the CHART program to determine whether public reporting of ICU mortality rates improved patient outcomes (4). Using Medicare claims, the authors compared mortality rates and discharge patterns among patients cared for in California ICUs in the 2 years before and after the program began. Using adjacent states or matched hospital referral regions as controls, they performed a “difference-in-difference” analysis to isolate the effect of the intervention from other regional and national trends in outcomes over the study period (5). This approach is the study’s major strength, as it allowed the authors to subtract out temporal changes in outcomes in control states from those in California, increasing one’s confidence that any measured change in outcomes was a result of public reporting. The authors found that public reporting of ICU mortality rates for California hospitals had no effect on actual hospital or 30-day mortality rates. This study is the first to demonstrate that the public reporting of an ICU quality measure had no effect on patient outcomes, and adds to a growing body of evidence suggesting that public reporting in isolation is an ineffective way to improve patient care (6). It is not surprising that public reporting failed to improve outcomes in critically ill patients when one considers the major mechanisms through which public reporting is thought to improve care. In theory, public reporting allows patients to make more informed choices when selecting where to receive care, thus increasing the market share of high-quality hospitals. But it is difficult to imagine a critically ill patient being able to act on publically reported data, negating consumer choice as plausible mechanism. Payers in California also did not reward hospitals that had lower ICU mortality rates, which is another potentially powerful way to incentivize change. Together, these issues may have reduced the ability of the CHART program to affect ICU outcomes. Selective transfer of critically ill patients to high-quality hospitals, based on public available data, is perhaps another mechanism for public reporting to improve outcomes for critically ill patients (7). The authors found an increase in transfer rates of ICU patients between California hospitals in the years after the implementation of the CHART program when compared with control in states. However, the decision-making surrounding the transfer of a critically ill patient is likely minimally dependent on the receiving hospital’s risk-adjusted mortality (8). Physicians may also view a hospital’s risk-adjusted mortality as inaccurate (9), making it irrelevant to any decision about transfer. Thus, it is unlikely that transfer decisions were based on the publicly reported outcomes data of the receiving hospital. An alternative explanation for the increase in transfer rates observed by the authors is that hospitals were inappropriately transferring patients likely to die, with the intent of lowering their own hospital’s mortality rate. Unintended consequences are common and important to recognize in any performance measurement program, and they often can harm the most vulnerable hospitals and patients (10, 11). Perhaps most important, the study by Reineck and colleagues exemplifies the potential power of an empirical evaluation of a quality improvement policy. Although CHART was dissolved because of lack of payer buy-in, the study confirms that the CHART program did not have the intended effect of improving mortality, further justifying its abandonment. Moreover, these data suggest that new programs revolving around public reporting of ICU mortality may fail to provide adequate improvements in health for the investment. When program evaluation can be performed in a timely fashion, it has the ability to motivate dramatic change in regional or national quality measurement programs. For example, after learning there was no significant reduction in complications after implementation of a CMS policy to restrict coverage of bariatric surgical procedures to hospitals designated as centers of excellence for bariatric surgery, CMS dropped the policy (12). In the ICU, using similar approaches as in the study by Reineck and colleagues, empirical evaluation of forthcoming regional or national quality measurement policies may be equally informative. For example, the growing incidence of sepsis and interest in improving sepsis care has resulted in several policies aimed at improving care for this condition. In 2013, hospitals in New York State were required to develop and report their protocols for identification and management of patients with sepsis to the state’s department of health (13). In 2017, CMS will mandate that hospitals begin reporting adherence to the National Quality Form severe sepsis and septic shock bundle (14). These are only two of several forthcoming regional or national programs targeting patients with critical illness that must be subjected to a similarly rigorous evaluation to ensure they are effectively improving the care of such patients without increasing harms. Because of ongoing concerns regarding the high costs and wide variation in the quality of healthcare, the performance measurement enterprise is only likely to grow. The research community must actively participate in the development, implementation, and evaluation of performance measurement programs (15, 16). As these programs move from public reporting to pay-for-performance as their primary mechanism for incentivizing improvement, the research community must ensure they are both fair and effective. Investigators can turn to studies such as that by Reineck and colleagues as an example of how it should be done.

Objective:Influenza is the most common vaccine-preventable disease in the United States; however, little is known about the burden of critical illness due to influenza virus infection. Our primary objective was to estimate the proportion of all critical illness hospitalizations that are attributable

The last decade has witnessed significant improvements in the care of acutely ill hospitalized patients. Elderly patients with an acute myocardial infarction (AMI) are now nearly twice as likely to receive evidence-based care and one-third less likely to die during their hospital stay compared to just ten years ago1,2. Similar trends exist for congestive heart failure (CHF) and pneumonia2. National public reporting and pay for performance (P4P) efforts, such as those implemented by the Centers for Medicare and Medicaid services (CMS), have contributed to improvements in care for these conditions1. Since those CMS programs were developed, however, the epidemiology of hospital care has changed in the United States. Improvements in outpatient care have substantially reduced the incidence of hospitalization for the conditions on which CMS currently focuses, specifically AMI, CHF, and pneumonia. Moreover, fewer than 10% of patients with these three conditions now die during their hospital stay. Meanwhile, among inpatients, sepsis has become the most prevalent and most costly disease, and is associated with high in-hospital mortality rate3,4. The time has come for CMS to explicitly develop new quality mandates focused on sepsis. By adding a specific focus on sepsis, CMS might achieve greater improvements in patient outcomes and advance the quality of hospital-based care. Sepsis is now the most common non-pregnancy related primary discharge diagnosis for Medicare and Medicaid, and in the top five for private payers2,3. These numbers underestimate the total burden of sepsis, as many patients admitted with other common diagnoses also have sepsis or develop sepsis during the hospital stay. With an increasing incidence and high case-fatality rate, sepsis now accounts for nearly half of all hospital deaths4. Yet, hospitals vary widely in their adherence to sepsis guidelines5. In marked contrast to adherence to treatment guidelines for AMI, CHF and pneumonia process measures, published reports routinely cite less than 35% compliance with current best practice for sepsis care5. Some may suggest that the rollout of national accountability measures for sepsis is premature6. These concerns, however, are largely not empirically based. Some contend that claims-based definitions of sepsis lack adequate validity, but in fact the specificity for the most widely used claims-based definition for severe sepsis is greater than 96% and is comparable to that of other CMS conditions7. There are concerns that the increase in sepsis diagnoses may reflect “upcoding” (selection of billing codes intended to increase reimbursement) rather than a true change in incidence; mandates could improve the identification and appropriate care of true sepsis cases, and as an additional effect could allow more accurate epidemiology and policy monitoring. While there are concerns about whether outcomes for patients with sepsis can be improved, RCTs and observational studies have identified early recognition, timely antibiotic administration, and aggressive fluid resuscitation as effective in reducing sepsis-related mortality8. Moreover, sizeable improvements in mortality in sepsis can be achieved through quality initiatives that integrate these therapies9, in part because too few patients currently receive optimal care. A next-generation quality improvement target like sepsis will require implementation of next-generation performance measures that build on what has been learned since public reporting and P4P began. Simply mandating reporting of 30-day risk-adjusted mortality, or even CMS’ recent decision to require hospitals to report adherence to the NQF’s severe sepsis and septic shock management bundle,10 may help, but will not be sufficient. In this, sepsis is not unique; other patients could benefit from more innovative approaches too. However, for CMS to maximally drive performance improvement, these changes are particularly important for sepsis. First, new mandates to improve sepsis must address the reality that sepsis is frequently under-diagnosed7. Existing quality metrics for AMI, CHF and pneumonia capitalized on previous decades’ work to make accurate recognition of those conditions nearly universal. But quality improvement cannot be limited to only those conditions for which the diagnosis is already accurate. Quality improvement mandates could improve not only the completeness, but also the precision of sepsis diagnosis. A quality improvement focus on diagnostic accuracy could provide spillover benefits for non-sepsis patients as well. New sepsis mandates have the potential to advance the science and practice of quality improvement to face the common reality of diagnostic ambiguity or inadequacy. Second, sepsis mandates should focus on catalyzing and aggregating local efforts for quality improvement. Increasing evidence suggests that current public reporting and P4P methods are insufficient tools to fully improve care11. Instead, CMS could scale up collaborative quality improvement using methods that incentivize gains in the culture of care, foster professionalism and sharing of best practices, and improve workflow of care processes—that is, work to target learning rather than only judging11. In doing so, CMS could promote sharing across hospital boundaries of how to better care for patients. This is particularly important for sepsis given the need to assess timely recognition10—whereas traditional mandates involving financial penalties would create perverse incentives to hide delayed diagnosis of sepsis, rather than to fix the problem. Third, sepsis mandates should plan for phased implementation, improving the measures in select sites prior to national rollout. CMS has developed an Innovation Center as an infrastructure for the development and testing of healthcare payment and service delivery models. Alternatively, regional quality collaboratives, such as state wide multi-hospital networks, could be contracted by CMS to examine benefits and harms of specific sepsis mandates. Early adopters serve as laboratories for refining measurement and averting unintended consequences. Such early adopters can include those just beginning quality improvement, as well as longstanding leaders such as Kaiser Permanente and Intermountain Healthcare. Careful assessment of the challenges of implementing policy mandates in these settings brings to light the potential for harms when mandates are widely implemented, but does so in a way that helps develop solutions rather than simply documenting barriers. Fourth, sepsis mandates must plan to be highly responsive to new evidence. Quality measurement seeks to ensure that every patient receives the currently recognized best possible care; therefore, when best care improves, quality measurement should improve in tandem. One NQF sepsis measure provides an optimistic example of such responsiveness: the ProCESS trial demonstrated that focus on recognition, early antibiotics, and fluid resuscitation may achieve equivalent sepsis outcomes to more complicated protocols that mandate central line placement8. NQF promptly revised its measure to remove the need for central venous catheterization. A new sepsis mandate must be equally responsive to new, yet-to-be-published evidence. Pragmatically this means planning for regular review of measures, having a system for putting some measures “on hiatus” pending a review of new data, and admitting that a single promulgation will not forever capture the state-of-the-art of a dynamic science. Sepsis is a major public health problem and has become a dominant diagnosis and cause of death in US hospitals. Implementation of national programs that track and mandate improvements in care and outcomes of sepsis could improve the prognosis for patients with sepsis. National programs for sepsis are needed that improve imperfect diagnosis, that focus on learning rather than judging, that use phased implementation, and that have planned growth in response to new scientific evidence rather than static rules. Current practices mean that only one third of patients with sepsis will receive the excellent care. Clinicians--as well as the health care system--can do better if properly led, and now is the time to start to do so.

Many health care performance measures are either not based on high-quality clinical evidence or not tightly linked to patient-centered outcomes, limiting their usefulness in quality improvement. In this report we summarize the proceedings of an American Thoracic Society workshop convened to address this problem by reviewing current approaches to performance measure development and creating a framework for developing high-quality performance measures by basing them directly on recommendations from well-constructed clinical practice guidelines. Workshop participants concluded that ideally performance measures addressing care processes should be linked to clinical practice guidelines that explicitly rate the quality of evidence and the strength of recommendations, such as the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) process. Under this framework, process-based performance measures would only be developed from strong recommendations based on high- or moderate-quality evidence. This approach would help ensure that clinical processes specified in performance measures are both of clear benefit to patients and supported by strong evidence. Although this approach may result in fewer performance measures, it would substantially increase the likelihood that quality-improvement programs based on these measures actually improve patient care.

Early identification of nontrauma patients in need of critical care services in the emergency setting may improve triage decisions and facilitate regionalization of critical care.To determine the out-of-hospital clinical predictors of critical illness and to characterize the performance of a simple score for out-of-hospital prediction of development of critical illness during hospitalization.Population-based cohort study of an emergency medical services (EMS) system in greater King County, Washington (excluding metropolitan Seattle), that transports to 16 receiving facilities.Nontrauma, non-cardiac arrest adult patients transported to a hospital by King County EMS from 2002 through 2006. Eligible records with complete data (N = 144,913) were linked to hospital discharge data and randomly split into development (n = 87,266 [60%]) and validation (n = 57,647 [40%]) cohorts.Development of critical illness, defined as severe sepsis, delivery of mechanical ventilation, or death during hospitalization.Critical illness occurred during hospitalization in 5% of the development (n = 4835) and validation (n = 3121) cohorts. Multivariable predictors of critical illness included older age, lower systolic blood pressure, abnormal respiratory rate, lower Glasgow Coma Scale score, lower pulse oximetry, and nursing home residence during out-of-hospital care (P.01 for all). When applying a summary critical illness prediction score to the validation cohort (range, 0-8), the area under the receiver operating characteristic curve was 0.77 (95% confidence interval [CI], 0.76-0.78), with satisfactory calibration slope (1.0). Using a score threshold of 4 or higher, sensitivity was 0.22 (95% CI, 0.20-0.23), specificity was 0.98 (95% CI, 0.98-0.98), positive likelihood ratio was 9.8 (95% CI, 8.9-10.6), and negative likelihood ratio was 0.80 (95% CI, 0.79- 0.82). A threshold of 1 or greater for critical illness improved sensitivity (0.98; 95% CI, 0.97-0.98) but reduced specificity (0.17; 95% CI, 0.17-0.17).In a population-based cohort, the score on a prediction rule using out-of-hospital factors was significantly associated with the development of critical illness during hospitalization. This score requires external validation in an independent population.

Background Despite widespread guidelines recommending the use of lung-protective ventilation (LPV) in patients with acute lung injury (ALI), many patients do not receive this lifesaving therapy. We sought to estimate the incremental clinical and economic outcomes associated with LPV and determined the maximum cost of a hypothetical intervention to improve adherence with LPV that remained cost-effective. Methods Adopting a societal perspective, we developed a theoretical decision model to determine the cost-effectiveness of LPV compared to non-LPV care. Model inputs were derived from the literature and a large population-based cohort of patients with ALI. Cost-effectiveness was determined as the cost per life saved and the cost per quality-adjusted life-years (QALYs) gained. Results Application of LPV resulted in an increase in QALYs gained by 15% (4.21 years for non-LPV vs 4.83 years for LPV), and an increase in lifetime costs of $7,233 per patient with ALI ($99,588 for non-LPV vs $106,821 for LPV). The incremental cost-effectiveness ratios for LPV were $22,566 per life saved at hospital discharge and $11,690 per QALY gained. The maximum, cost-effective, per patient investment in a hypothetical program to improve LPV adherence from 50 to 90% was $9,482. Results were robust to a wide range of economic and patient parameter assumptions. Conclusions Even a costly intervention to improve adherence with low-tidal volume ventilation in patients with ALI reduces death and is cost-effective by current societal standards.

Introduction Prompt treatment of severe sepsis in the Emergency Department reduces deaths, but the role of prehospital fluid resuscitation is unknown. We sought to determine the risk-adjusted association between prehospital fluid administration and hospital mortality among emergency medical services (EMS) patients admitted with severe sepsis. Methods We performed a prospective, observational study of patients hospitalized with severe sepsis on admission among 45,394 adult EMS encounters taken to 15 hospitals from 11/2009 to 12/2010 by a two-tier EMS system in King County, Washington. The region mandated recording of prehospital intravenous catheter and fluid administration in prehospital records, along with detailed demographic, incident, physiologic, and hospital adjustment variables. We determined the effect of prehospital intravenous catheter or fluid versus no catheter or fluid on all-cause mortality using multivariable logistic regression. Results Of all encounters, 1,350 met criteria for severe sepsis on admission, of whom 205 (15%) died by hospital discharge, 312 (23%) received prehospital intravenous fluid, 90 (7%) received a prehospital catheter alone and 948 (70%) did not receive catheter or fluid. EMS administered a median prehospital fluid volume of 500 mL (interquartile range (IQR): 200, 1000 mL). In adjusted models, the administration of any prehospital fluid was associated with reduced hospital mortality (Odds ratio =0.46; 95% Confidence interval: 0.23, 0.88; P =0.02) compared to no prehospital fluid. The odds of hospital mortality were also lower among severe sepsis patients treated with prehospital intravenous catheter alone (Odds ratio =0.3; 95% Confidence interval: 0.17 to 0.57; P

Race has been identified as an important risk factor for the development of sepsis and as a predictor of poor outcomes in sepsis. For example, black individuals have been demonstrated to be nearly twice as likely to develop sepsis and to have greater mortality from sepsis than white individuals. Recent data from a longitudinal cohort, which examined incident hospitalizations for infections occurring among participants in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) cohort, contradicts this prior research. Investigators determined that black participants were significantly less likely than white participants to present to the hospital with either infection or sepsis. Although these results are intriguing, they highlight our inadequate understanding of the relationship between race and sepsis and motivate the need for higher quality epidemiologic research to isolate the true role of race in the development of sepsis.

Background To determine the time and temperature stability of whole blood lactate using experimental conditions applicable to the out-of-hospital environment. Findings We performed a prospective, clinical laboratory-based study at an academic hospital. Whole blood lactate was obtained by venipuncture from five post-prandial, resting subjects. Blood was stored in lithium heparinized vacutainers in three temperature conditions: 1) room temperature (20°C), 2) wrapped in a portable, instant ice pack (0°C), or 3) wet ice (0°C). Lactate concentrations (mmol/L) were measured at 0, 5, 10, 20, and 30 minutes after sampling, and compared using repeated measures analysis of variance. Mean baseline lactate among resting subjects (N = 5) was 1.24 mmol/L (95%CI: 0.49,1.98 mmol/L). After 30 minutes, lactate concentration increased, on average, by 0.08 mmol/L (95%CI: 0.02,0.13 mmol/L), 0.18 mmol/L (95%CI: 0.07,0.28 mmol/L), and 0.36 mmol/L (95%CI: 0.24,0.47 mmol/L) when stored in wet ice, ice pack, and room temperature, respectively. The increase in lactate was similar in samples wrapped in portable ice pack or stored in wet ice at all time points (p > 0.05), and met criteria for equivalence at 30 minutes. However, lactate measurements from whole blood stored at room temperature were significantly greater, on average, than wet ice or portable ice pack within five and ten minutes, respectively (p < 0.05). Conclusions Whole blood lactate measurements using samples stored in a portable ice pack are similar to wet ice for up to 30 minutes. These conditions are applicable to the out-of-hospital environment, and should inform future studies of pre-hospital measurement of lactate.

Introduction Little is known about the mechanisms through which intensivist physician staffing influences patient outcomes. We aimed to assess the effect of closed-model intensive care on evidence-based ventilatory practice in patients with acute lung injury (ALI). Methods We conducted a secondary analysis of a prospective population-based cohort of 759 patients with ALI who were alive and ventilated on day three of ALI, and were cared for in 23 intensive care units (ICUs) in King County, Washington. Results We compared day three tidal volume (VT) in open versus closed ICUs adjusting for potential patient and ICU confounders. In 13 closed model ICUs, 429 (63%) patients were cared for. Adjusted mean VT (mL/Kg predicted body weight (PBW) or measured body weight if height not recorded) for patients in closed ICUs was 1.40 mL/Kg PBW (95% confidence interval (CI) = 0.57 to 2.24 mL/Kg PBW) lower than patients in open model ICUs. Patients in closed ICUs were more likely (odds ratio (OR) = 2.23, 95% CI = 1.09 to 4.56) to receive lower VT (≤ 6.5 mL/Kg PBW) and were less likely (OR = 0.30, 95% CI = 0.17 to 0.55) to receive a potentially injurious VT (≥ 12 mL/Kg PBW) compared with patients cared for in open ICUs, independent of other covariates. The effect of closed ICUs on hospital mortality was not changed after accounting for delivered VT. Conclusions Patients with ALI cared for in closed model ICUs are more likely to receive lower VT and less likely to receive higher VT, but there were no other differences in measured processes of care. Moreover, the difference in delivered VT did not completely account for the improved mortality observed in closed model ICUs.