Search

Objective: Healthcare accreditation surveyors are well positioned to gain access to hospitals and apply their existing data collection skills to research. Consequently, we contracted and trained a surveyor cohort to collect research data for the Deepening our Understanding of Quality in Australia (DUQuA) project. The aim of this study is to explore and compare surveyors' perceptions and experiences in collecting quality and safety data for accreditation and for health services research.Design: A qualitative, comparative study.Setting and Participants: Ten surveyors participated in semi-structured interviews, which were audio recorded, transcribed and coded using Nvivo11. Interview transcripts of participants were analysed thematically and separately, providing an opportunity for comparison and for identifying common themes and subthemes.Intervention(s): None.Main Outcome Measure(s): Topics addressed data collection for healthcare accreditation and research, including preparation and training, structure, organization, attitudes and behaviours of staff and perceptions of their role.Results: Five themes and ten subthemes emerged from the interviews: (1) overlapping facilitators for accreditation and research data collection, (2) accreditation-specific facilitators, (3) overlapping barriers for accreditation and research data collection, (4) research data collection-specific barriers and (5) needs and recommendations. Subthemes were (1.1) preparation and training availability, (1.2) prior knowledge and experiences; (2.1) ease of access, (2.2) high staff engagement, (3.1) time, (4.1) poor access and structure, (4.2) lack of staff engagement, (4.3) organizational changes; (5.1) short-notice accreditation and (5.2) preparation for future research.Conclusions: Although hospital accreditation and research activities require different approaches to data collection, we found that suitably trained accreditation surveyors were able to perform both activities effectively. The barriers surveyors encountered when collecting data for research provide insight into the challenges that may be faced when visiting hospitals for short-notice accreditation. [ABSTRACT FROM AUTHOR]

Healthcare organisations vary in the degree to which they implement quality and safety systems and strategies. Large-scale cross-sectional studies have been implemented to explore whether this variation is associated with outcomes relevant at the patient level. The Deepening our Understanding of Quality in Australia (DUQuA) study draws from earlier research of this type, to examine these issues in 32 Australian hospitals. This paper outlines the key implementation and analysis challenges faced by DUQuA. Many of the logistical difficulties of implementing DUQuA derived from compliance with the administratively complex and time-consuming Australian ethics and governance system designed principally to protect patients involved in clinical trials, rather than for low-risk health services research. The complexity of these processes is compounded by a lack of organizational capacity for multi-site health services research; research is expected to be undertaken in addition to usual work, not as part of it. These issues likely contributed to a relatively low recruitment rate for hospitals (41% of eligible hospitals). Both sets of issues need to be addressed by health services researchers, policymakers and healthcare administrators, if health services research is to flourish. Large-scale research also inevitably involves multiple measurements. The timing for applying these measures needs to be coherent, to maximise the likelihood of finding real relationships between quality and safety systems and strategies, and patient outcomes; this timing was less than ideal in DUQuA, in part due to administrative delays. Other issues that affected our study include low response rates for measures requiring recruitment of clinicians and patients, missing data and a design that necessarily included multiple statistical comparisons. We discuss how these were addressed. Successful completion of these projects relies on mutual and ongoing commitment, and two-way communication between the research team and hospital staff at all levels. This will help to ensure that enthusiasm and engagement are established and maintained. [ABSTRACT FROM AUTHOR]

Objective: The aim of this study was to develop and refine indices to measure organization and care pathway-level quality management systems in Australian hospitals.Design: A questionnaire survey and audit tools were derived from instruments validated as part of the Deepening Our Understanding of Quality improvement in Europe (DUQuE) study, adapted for Australian hospitals through expert opinion. Statistical processes were used to explore the factor structure, reliability and non-redundancy and descriptive statistics of the scales.Setting: Thirty-two large Australian public hospitals.Participants: Audit of quality management processes at organization-level and care pathway processes at department level for three patient conditions (acute myocardial infarction (AMI), hip fracture and stroke) and senior quality manager, at each of the 32 participating hospitals.Main Outcome Measure(s): The degree of quality management evident at organization and care pathway levels.Results: Analysis yielded seven quality systems and strategies scales. The three hospital-level measures were: the Quality Management Systems Index (QMSI), the Quality Management Compliance Index (QMCI) and the Clinical Quality Implementation Index (CQII). The four department-level measures were: Specialised Expertise and Responsibility (SER), Evidence-Based Organisation of Pathways (EBOP), Patient Safety Strategies (PSS) and Clinical Review (CR). For QMCI, and for seven out of eight subscales in QMSI, adequate internal consistency (Cronbach's $\alpha$ >0.8) was achieved. For CQII, lack of variation and ceiling effects in the data resulted in very low internal consistency scores, but items were retained for theoretical reasons. Internal consistency was high for CR (Cronbach's $\alpha$ 0.74-0.88 across the three conditions), and this was supported by all item-total correlations exceeding the desired threshold. For EBOP, Cronbach's $\alpha$ was acceptable for hip fracture (0.80) and stroke (0.76), but only moderate for AMI (0.52). PSS and SER scales were retained for theoretical reasons, although internal consistencies were only moderate (SER) to poor (PSS).Conclusions: The Deepening our Understanding of Quality in Australia (DUQuA) organization and department scales can be used by Australian hospital managers to assess and measure improvement in quality management at organization and department levels within their hospitals and are readily modifiable for other health systems depending on their needs. [ABSTRACT FROM AUTHOR]

This paper examines the principles of benchmarking in healthcare and how benchmarking can contribute to practice improvement and improved health outcomes for patients. It uses the Deepening our Understanding of Quality in Australia (DUQuA) study published in this Supplement and DUQuA's predecessor in Europe, the Deepening our Understanding of Quality improvement in Europe (DUQuE) study, as models. Benchmarking is where the performances of institutions or individuals are compared using agreed indicators or standards. The rationale for benchmarking is that institutions will respond positively to being identified as a low outlier or desire to be or stay as a high performer, or both, and patients will be empowered to make choices to seek care at institutions that are high performers. Benchmarking often begins with a conceptual framework that is based on a logic model. Such a framework can drive the selection of indicators to measure performance, rather than their selection being based on what is easy to measure. A Donabedian range of indicators can be chosen, including structure, process and outcomes, created around multiple domains or specialties. Indicators based on continuous variables allow organizations to understand where their performance is within a population, and their interdependencies and associations can be understood. Benchmarking should optimally target providers, in order to drive them towards improvement. The DUQuA and DUQuE studies both incorporated some of these principles into their design, thereby creating a model of how to incorporate robust benchmarking into large-scale health services research. [ABSTRACT FROM AUTHOR]

Objective: This study aimed to explore the associations between the organization-level quality arrangements, improvement and implementation and department-level safety culture and leadership measures across 32 large Australian hospitals.Design: Quantitative observational study, using linear and multi-level modelling to identify relationships between quality management systems and clinician safety culture and leadership.Setting: Thirty-two large Australian public hospitals.Participants: Quality audit at organization level, senior quality manager at each participating hospital, 1382 clinicians (doctors, nurses and allied health professionals).Main Outcome Measures: Associations between organization-level quality measures and department-level clinician measures of teamwork climate, safety climate and leadership for acute myocardial infarction (AMI), hip fracture and stroke treatment conditions.Results: We received 1332 valid responses from participants. The quality management systems index (QMSI, a questionnaire-based measure of the hospitals' quality management structures) was 'positively' associated with all three department-level scales in the stroke department, with safety culture and leadership in the emergency department, but with none of the three scales in the AMI and hip fracture departments. The quality management compliance index (QMCI, an external audit-based measure of the quality improvement activities) was 'negatively' associated with teamwork climate and safety climate in AMI departments, after controlling for QMSI, but not in other departments. There was no association between QMCI and leadership in any department, after controlling for QMSI, and there was no association between the clinical quality implementation index (CQII, an external audit-based measure of the level of implementation of quality activities) and any of the three department-level scales in any of the four departments, after controlling for both QMSI and QMCI.Conclusions: The influence of organization-level quality management systems on clinician safety culture and leadership varied depending on the hospital department, suggesting that whilst there was some consistency on patient safety attitudes and behaviours throughout the organizations, there were also other factors at play. [ABSTRACT FROM AUTHOR]

Objective: This study aimed to refine and validate a tool to measure safety culture and leadership in Australian hospitals.Design: The clinician safety culture and leadership questionnaire was constructed by combining and refining the following two previously validated scales: Safety Attitudes Questionnaire and the Leadership Effectiveness Survey. Statistical processes were used to explore the factor structure, reliability, validity and descriptive statistics of the new instrument.Setting: Thirty-two large Australian public hospitals.Participants: 1382 clinicians (doctors, nurses and allied health professionals).Main Outcome Measure(s): Descriptive statistics, structure and validity of clinician safety culture and leadership scale.Results: We received 1334 valid responses from participants. The distribution of ratings was left-skewed, with a small ceiling effect, meaning that scores were clustered toward the high end of the scale. Using confirmatory factor analysis, we confirmed the structure of the three scales as a combined measure of safety culture and leadership. The data were divided into equal calibration and validation datasets. For the calibration dataset, the Chi-square: df ratio was 4.4, the root mean square error of approximation RMSEA (a measure of spread of the data) was 0.071, the standardized root mean square residual SRMR (an absolute measure of the fit of the data) was 0.058 and the Confirmatory Fit Index (CFI) (another test confirming the fit of the data) was 0.82; while none of the indices suggested good fit, all but CFI fell within acceptable thresholds. All factors demonstrated adequate internal consistency and construct reliability, as desired. All three domains achieved discriminant validity through cross-loadings, meaning that the three domains were determined to be independent constructs. Results for the validation dataset were effectively identical to those found in the calibration dataset.Conclusions: While the model may benefit from additional refinement, we have validated the tool for measuring clinician safety culture and leadership in our Australian sample. The DUQuA safety culture and leadership scale can be used by Australian hospitals to assess clinician safety culture and leadership, and is readily modifiable for other health systems depending on their needs. [ABSTRACT FROM AUTHOR]

Objective: We aimed to examine whether Emergency Department (ED) quality strategies, safety culture and leadership were associated with patient-level outcomes, after controlling for other organization-level factors, in 32 large Australian hospitals.Design: Quantitative observational study, using linear and multi-level modelling to identify relationships between quality management systems at organization level; quality strategies at ED level for acute myocardial infarction (AMI), hip fracture and stroke; clinician safety culture and leadership and patient-level outcomes of waiting time and length of stay.Setting: Thirty-two large Australian public hospitals.Participants: Audit of quality management processes at organization and ED levels, senior quality manager at each of the 32 participating hospitals, 394 ED clinicians (doctors, nurses and allied health professionals).Main Outcome Measure(s): Within the multi-level model, associations were assessed between organization-level quality measures and ED quality strategies; organization-level quality measures and ED quality strategies and ward-level clinician measures of teamwork climate (TC), safety climate (SC) and leadership for AMI, hip fracture and stroke treatment conditions; and organization-level quality measures and ED quality strategies and ward-level clinician measures of TC, SC and leadership, and ED waiting time and length of stay (performance).Results: We found seven statistically significant associations between organization-level quality systems and ED-level quality strategies; four statistically significant associations between quality systems and strategies and ED safety culture and leadership; and nine statistically significant associations between quality systems and strategies and ED safety culture and leadership, and ED waiting time and length of stay.Conclusions: Organization-level quality structures influence ED-level quality strategies, clinician safety culture and leadership and, ultimately, waiting time and length of stay for patients. By focusing only on time-based measures of ED performance we risk punishing EDs that perform well on patient safety measures. We need to better understand the trade-offs between implementing safety culture and quality strategies and improving patient flow in the ED, and to place more emphasis on other ED performance measures in addition to time. [ABSTRACT FROM AUTHOR]

Objective: Little is known about the influence that hospital quality systems have on quality at department level, in Australia and elsewhere. This study assessed the relationships between organizational-level quality management systems, and the extent to which hospital-level quality management systems and department-level quality management strategies are related.Design: A multi-level, cross-sectional, mixed-method study.Setting and Participants: As part of the Deepening our Understanding of Quality in Australia (DUQuA) project, we invited all large hospitals in Australia (~200 or more beds) which provided acute myocardial infarction (AMI), hip fracture and stroke care. The quality managers of these hospitals were the respondents for one of seven measures of hospital quality management systems and strategies. Data across the six remaining measures were collected through site visits by external surveyors assessing the participating hospitals.Main Outcome Measures: Relationships were assessed between three organization-level quality management system measures: a self-report measure assessing organization-level quality activities (quality management systems index, QMSI); externally assessed organization-level compliance to procedures used to plan, monitor and improve quality of care (quality management compliance index, QMCI); and externally assessed implementation of quality systems (clinical quality implementation index, CQII). Associations were also assessed between organization-level quality management systems and department-level quality management strategies: how clinical responsibilities are assigned for a particular condition; whether department organization processes are organized to facilitate evidence-based care recommendations; compliance with selected recommendations of international agencies; and whether clinical reviews are performed systematically.Results: Of 78 invited hospitals, 32 participated in the study. QMSI was positively associated with QMCI and CQII, but after controlling for QMSI, no relationship was found between QMCI and CQII. There appears to be a cluster of relationships between QMSI and department-level measures, but this was not consistent across all departments.Conclusion: This is the first national study undertaken in Australia to assess relationships within and between organization-level and department-level quality management systems. These quality management system tools align with many components of accreditation standards and may be useful for hospitals in continuously monitoring and driving improvement. [ABSTRACT FROM AUTHOR]

With this paper, we initiate the Supplement on Deepening our Understanding of Quality in Australia (DUQuA). DUQuA is an at-scale, cross-sectional research programme examining the quality activities in 32 large hospitals across Australia. It is based on, with suitable modifications and extensions, the Deepening our Understanding of Quality improvement in Europe (DUQuE) research programme, also published as a Supplement in this Journal, in 2014. First, we briefly discuss key data about Australia, the health of its population and its health system. Then, to provide context for the work, we discuss previous activities on the quality of care and improvement leading up to the DUQuA studies. Next, we present a selection of key interventional studies and policy and institutional initiatives to date. Finally, we conclude by outlining, in brief, the aims and scope of the articles that follow in the Supplement. This first article acts as a framing vehicle for the DUQuA studies as a whole. Aggregated, the series of papers collectively attempts an answer to the questions: what is the relationship between quality strategies, both hospital-wide and at department level? and what are the relationships between the way care is organised, and the actual quality of care as delivered? Papers in the Supplement deal with a multiplicity of issues including: how the DUQuA investigators made progress over time, what the results mean in context, the scales designed or modified along the way for measuring the quality of care, methodological considerations and provision of lessons learnt for the benefit of future researchers. [ABSTRACT FROM AUTHOR]

Objective: Patients can provide a unique perspective on the safety of care in hospitals. Understanding that the extent to which the way hospitals are organized for quality and safety is associated with patient perceptions of care is becoming increasingly valued and necessary for the direction of targeted interventions across healthcare systems. The UK-developed patient measure of safety (PMOS) assesses eight domains of ward safety from the patient point of view and has recently been adapted and piloted in Australia. The aim of this study is to test the psychometric properties of PMOS-Australia (PMOS-A) amongst a large cohort of hospitalized patients.Design: Cross-sectional questionnaire validation assessment.Setting and Participants: As part of the DUQuA project, the PMOS-A survey was distributed within acute myocardial infarction, hip fracture and stroke departments across 32 large public hospitals in Australia. Patients could complete the PMOS-A independently, or request the assistance of a family member/guardian, or staff on the wards-space was included to record mode of completion.Main Outcome Measures: Confirmatory factor analysis (CFA) was undertaken on a calibration sample to generate the model, and a validation sample was used to cross-validate the model. A subset of only those participants who received assistance for PMOS-A completion was also tested using CFA on a calibration and validation sample. Model fit indices (chi-square to degrees of freedom ratio [Chi-square:DF], root mean square error of approximation [RMSEA], comparative fit indices [CFI], standardized root mean squared residual [SRMR]), Cronbach's α, average inter-item correlations, construct reliability and cross-loadings were examined with reference to recommended thresholds to establish the extent of convergent validity and discriminant validity. A marker of criterion validity was assessed through testing associations between the PMOS-A and adherence to clinical guidelines.Results: Across the calibration and validation samples of the full (N = 911) and assisted completers only subset (N = 490), three (Chi-square:DF, SRMR, RMSEA) of the four indices consistently or almost always met thresholds for acceptable model fit. CFI indices did not meet the recommended limits (0.72-0.78, against a target > 0.9). Positive relationships were found for all tests between PMOS-A and adherence to clinical guidelines, and these were significant when assessed in the calibration datasets for the full and assisted completion samples.Conclusion: A sufficiently reliable and valid measure of patient perceptions of safety has been developed. These findings should provide adequate support to justify the use of this measure to assess patient perceptions of safety in Australian hospitals and can be modified for use elsewhere. [ABSTRACT FROM AUTHOR]

Objective: The Deepening our Understanding of Quality in Australia (DUQuA) project is a multisite, multi-level, cross-sectional study of 32 of the largest hospitals in Australia. This overview examines relationships between (i) organization-level quality management systems and department-level quality management strategies and (ii) patient-level measures (clinical treatment processes, patient-reported perceptions of care and clinical outcomes) within Australian hospitals.Design: We examined hospital quality improvement structures, processes and outcomes, collecting data at organization, department and patient levels for acute myocardial infarction (AMI), hip fracture and stroke. Data sources included surveys of quality managers, clinicians and patients, hospital visits, medical record reviews and national databases. Outcomes data and patient admissions data were analysed. Relationships between measures were evaluated using multi-level models. We based the methods on the Deepening our Understanding of Quality Improvement in Europe (DUQuE) framework, extending that work in parts and customizing the design to Australian circumstances.Setting, Participants and Outcome Measures: The 32 hospitals, containing 119 participating departments, provided wide representation across metropolitan, inner and outer regional Australia. We obtained 31 quality management, 1334 clinician and 857 patient questionnaires, and conducted 2401 medical record reviews and 151 external assessments. External data via a secondary source comprised 14 460 index patient admissions across 14 031 individual patients. Associations between hospital, Emergency Department (ED) and department-level systems and strategies and five patient-level outcomes were assessed: 19 of 165 associations (11.5%) were statistically significant, 12 of 79 positive associations (15.2%) and 7 of 85 negative associations (8.2%).Results: We did not find clear relationships between hospital-level quality management systems, ED or department quality strategies and patient-level outcomes. ED-level clinical reviews were related to adherence to clinical practice guidelines for AMI, hip fracture and stroke, but in different directions. The results, when considered alongside the DUQuE results, are suggestive that front line interventions may be more influential than department-level interventions when shaping quality of care and that multi-pronged strategies are needed. Benchmark reports were sent to each participating hospital, stimulating targeted quality improvement activities.Conclusions: We found no compelling relationships between the way care is organized and the quality of care across three targeted patient-level outcome conditions. The study was cross-sectional, and thus we recommend that the relationships studied should be assessed for changes across time. Tracking care longitudinally so that quality improvement activities are monitored and fed back to participants is an important initiative that should be given priority as health systems strive to develop their capacity for quality improvement over time. [ABSTRACT FROM AUTHOR]