Search

Audit and feedback (A&F) is a commonly used quality improvement (QI) approach. A Cochrane review indicates that A&F is generally effective and leads to modest improvements in professional practice but with considerable variation in the observed effects. While we have some understanding of factors that enhance the effects of A&F, further research needs to explore when A&F is most likely to be effective and how to optimise it. To do this, we need to move away from two-arm trials of A&F compared with control in favour of head-to-head trials of different ways of providing A&F. This paper describes implementation laboratories involving collaborations between healthcare organisations providing A&F at scale, and researchers, to embed head-to-head trials into routine QI programmes. This can improve effectiveness while producing generalisable knowledge about how to optimise A&F. We also describe an international meta-laboratory that aims to maximise cross-laboratory learning and facilitate coordination of A&F research., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Patient and public involvement (PPI) is often an essential requirement for research funding. Distinctions can be drawn between clinical research, which generally focuses on patients, and implementation research, which generally focuses on health professional behaviour. There is uncertainty about the role of PPI in this latter field. We explored and defined the roles of PPI in implementation research to inform relevant good practice guidance., Methods: We used a structured consensus process using a convenience sample panel of nine experienced PPI and two researcher members. We drew on available literature to identify 21 PPI research roles. The panel rated their agreement with roles independently online in relation to both implementation and clinical research. Disagreements were discussed at a face-to-face meeting prior to a second online rating of all roles. Median scores were calculated and a final meeting held to review findings and consider recommendations., Results: Ten panellists completed the consensus process. For clinical research, there was strong support and consensus for the role of PPI throughout most of the research process. For implementation research, there were eight roles with consensus and strong support, seven roles with consensus but weaker support and six roles with no consensus. There were more disagreements relating to PPI roles in implementation research compared with clinical research. PPI was rated as contributing less to the design and management of implementation research than for clinical research., Conclusions: The roles of PPI need to be tailored according to the nature of research to ensure authentic and appropriate involvement. We provide a framework to guide the planning, conduct and reporting of PPI in implementation research, and encourage further research to evaluate its use., Competing Interests: Competing interests: None declared., (© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.)

There are many challenges in the development, implementation and evaluation of healthcare policy. One challenge is understanding how different stakeholders view a particular policy and what impact these views may have during implementation. Q-methodology is one approach that can be used to help policy makers and researchers actively engage with those who are important in policy implementation, and anticipate their responses. Q-methodology combines qualitative and quantitative research methods to systematically explore and describe the range of viewpoints about a topic. Participants are required to rank a set of predefined statements relating to the topic, according to their own viewpoint. Factor analytic techniques then identify people who are like-minded in the way they view the topic and enable areas of consensus and divergence in viewpoint to be clearly defined. This mapping of viewpoints allows those working in policy implementation to anticipate likely barriers and levers in implementing new policies., Competing Interests: Competing interests: None declared., (© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.)

Objective: Valid, reliable critical appraisal tools advance quality improvement (QI) intervention impacts by helping stakeholders identify higher quality studies. QI approaches are diverse and differ from clinical interventions. Widely used critical appraisal instruments do not take unique QI features into account and existing QI tools (eg, Standards for QI Reporting Excellence) are intended for publication guidance rather than critical appraisal. This study developed and psychometrically tested a critical appraisal instrument, the QI Minimum Quality Criteria Set (QI-MQCS) for assessing QI-specific features of QI publications., Methods: Approaches to developing the tool and ensuring validity included a literature review, in-person and online survey expert panel input, and application to empirical examples. We investigated psychometric properties in a set of diverse QI publications (N=54) by analysing reliability measures and item endorsement rates and explored sources of disagreement between reviewers., Results: The QI-MQCS includes 16 content domains to evaluate QI intervention publications: Organisational Motivation, Intervention Rationale, Intervention Description, Organisational Characteristics, Implementation, Study Design, Comparator Description, Data Sources, Timing, Adherence/Fidelity, Health Outcomes, Organisational Readiness, Penetration/Reach, Sustainability, Spread and Limitations. Median inter-rater agreement for QI-MQCS items was κ 0.57 (83% agreement). Item statistics indicated sufficient ability to differentiate between publications (median quality criteria met 67%). Internal consistency measures indicated coherence without excessive conceptual overlap (absolute mean interitem correlation=0.19). The critical appraisal instrument is accompanied by a user manual detailing What to consider, Where to look and How to rate., Conclusions: We developed a ready-to-use, valid and reliable critical appraisal instrument applicable to healthcare QI intervention publications, but recognise scope for continuing refinement., (Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/)

Background: The term continuous quality improvement (CQI) is often used to refer to a method for improving care, but no consensus statement exists on the definition of CQI. Evidence reviews are critical for advancing science, and depend on reliable definitions for article selection., Methods: As a preliminary step towards improving CQI evidence reviews, this study aimed to use expert panel methods to identify key CQI definitional features and develop and test a screening instrument for reliably identifying articles with the key features. We used a previously published method to identify 106 articles meeting the general definition of a quality improvement intervention (QII) from 9427 electronically identified articles from PubMed. Two raters then applied a six-item CQI screen to the 106 articles., Results: Per cent agreement ranged from 55.7% to 75.5% for the six items, and reviewer-adjusted intra-class correlation ranged from 0.43 to 0.62. 'Feedback of systematically collected data' was the most common feature (64%), followed by being at least 'somewhat' adapted to local conditions (61%), feedback at meetings involving participant leaders (46%), using an iterative development process (40%), being at least 'somewhat' data driven (34%), and using a recognised change method (28%). All six features were present in 14.2% of QII articles., Conclusions: We conclude that CQI features can be extracted from QII articles with reasonable reliability, but only a small proportion of QII articles include all features. Further consensus development is needed to support meaningful use of the term CQI for scientific communication.

Background: Differences in contexts (eg, policies, healthcare organisation characteristics) may explain variations in the effects of patient safety practice (PSP) implementations. However, knowledge of which contextual features are important determinants of PSP effectiveness is limited and consensus is lacking on a taxonomy of which contexts matter., Methods: Iterative, formal discussions were held with a 22-member technical expert panel composed of experts or leaders in patient safety, healthcare systems, and methods. First, potentially important contextual features were identified, focusing on five PSPs. Then, two surveys were conducted to determine the context likely to influence PSP implementations., Results: The panel reached a consensus on a taxonomy of four broad domains of contextual features important for PSP implementations: safety culture, teamwork and leadership involvement; structural organisational characteristics (eg, size, organisational complexity or financial status); external factors (eg, financial or performance incentives or PSP regulations); and availability of implementation and management tools (eg, training organisational incentives). Panelists also tended to rate specific patient safety culture, teamwork and leadership contexts as high priority for assessing their effects on PSP implementations, but tended to rate specific organisational characteristic contexts as high priority only for use in PSP evaluations. Panelists appeared split on whether specific external factors and implementation/management tools were important for assessment or only description., Conclusion: This work can guide research commissioners and evaluators on the contextual features of PSP implementations that are important to report or evaluate. It represents a first step towards developing guidelines on contexts in PSP implementation evaluations. However, the science of context measurement needs maturing.

Background: Logic and experience suggest that it is easier in some situations than in others to change behaviour and organisation to improve patient safety. Knowing which 'context factors' help and hinder implementation of different changes would help implementers, as well as managers, policy makers, regulators and purchasers of healthcare. It could help to judge the likely success of possible improvements, given the conditions that they have, and to decide which of these conditions could be modified to make implementation more effective., Methods: The study presented in this paper examined research to discover any evidence reported about whether or how context factors influence the effectiveness of five patient safety interventions., Results: The review found that, for these five diverse interventions, there was little strong evidence of the influence of different context factors. However, the research was not designed to investigate context influence., Conclusions: The paper suggests that significant gaps in research exist and makes proposals for future research better to inform decision-making.

Objective: Development of a coherent literature evaluating patient safety practices has been hampered by the lack of an underlying conceptual framework. The authors describe issues and choices in describing and classifying diverse patient safety practices (PSPs)., Methods: The authors developed a framework to classify PSPs by identifying and synthesising existing conceptual frameworks, evaluating the draft framework by asking a group of experts to use it to classify a diverse set of PSPs and revising the framework through an expert-panel consensus process., Results: The 11 classification dimensions in the framework include: regulatory versus voluntary; setting; feasibility; individual activity versus organisational change; temporal (one-time vs repeated/long-term); pervasive versus targeted; common versus rare events; PSP maturity; degree of controversy/conflicting evidence; degree of behavioural change required for implementation; and sensitivity to context., Conclusion: This framework offers a way to classify and compare PSPs, and thereby to interpret the patient-safety literature. Further research is needed to develop understanding of these dimensions, how they evolve as the patient safety field matures, and their relative utilities in describing, evaluating and implementing PSPs.

Theories provide a way of understanding and predicting the effects of patient safety practices (PSPs), interventions intended to prevent or mitigate harm caused by healthcare or risks of such harm. Yet most published evaluations make little or no explicit reference to theory, thereby hindering efforts to generalise findings from one context to another. Theories from a wide range of disciplines are potentially relevant to research on PSPs. Theory can be used in research to explain clinical and organisational behaviour, to guide the development and selection of PSPs, and in evaluating their implementation and mechanisms of action. One key recommendation from an expert consensus process is that researchers should describe the theoretical basis for chosen intervention components or provide an explicit logic model for 'why this PSP should work.' Future theory-driven evaluations would enhance generalisability and help build a cumulative understanding of the nature of change.

Background: The diversity of quality improvement interventions (QIIs) has impeded the use of evidence review to advance quality improvement activities. An agreed-upon framework for identifying QII articles would facilitate evidence review and consensus around best practices., Aim: To adapt and test evidence review methods for identifying empirical QII evaluations that would be suitable for assessing QII effectiveness, impact or success., Design: Literature search with measurement of multilevel inter-rater agreement and review of disagreement., Methods: Ten journals (2005-2007) were searched electronically and the output was screened based on title and abstract. Three pairs of reviewers then independently rated 22 articles, randomly selected from the screened list. Kappa statistics and percentage agreement were assessed. 12 stakeholders in quality improvement, including QII experts and journal editors, rated and discussed publications about which reviewers disagreed., Results: The level of agreement among reviewers for identifying empirical evaluations of QII development, implementation or results was 73% (with a paradoxically low kappa of 0.041). Discussion by raters and stakeholders regarding how to improve agreement focused on three controversial article selection issues: no data on patient health, provider behaviour or process of care outcomes; no evidence for adaptation of an intervention to a local context; and a design using only observational methods, as correlational analyses, with no comparison group., Conclusion: The level of reviewer agreement was only moderate. Reliable identification of relevant articles is an initial step in assessing published evidence. Advancement in quality improvement will depend on the theory- and consensus-based development and testing of a generalizable framework for identifying QII evaluations.

Clinical guidelines often make recommendations on the use of diagnostic tests. Compared with sensitivity and specificity, the use of pre- and post-test probabilities allows a more explicit and rational selection and interpretation of diagnostic tests. Ideally, clinical guidelines relating to diagnosis should routinely incorporate this information to enhance individualised decision making. We report our experience of incorporating pre- and post-test probabilities into a guideline on the investigation of women with postmenopausal bleeding developed by the Scottish Intercollegiate Guidelines Network. Issues relating to their application are highlighted, including the limitations of available evidence on diagnostic tests and prevalence of disease, acceptability to guideline users, and the uncertain impact on actual clinical decision making. Despite these potential difficulties, the incorporation of data on pre- and post-test probabilities into the development and presentation of guideline recommendations may offer an important opportunity to make clinical decision making more transparent for both clinicians and patients.