A practice improvement package at scale to improve management of birth asphyxia in Rwanda: a before-after mixed methods evaluation

Abstract Background Helping Babies Breathe (HBB) is a competency-based educational method for an evidence-based protocol to manage birth asphyxia in low resource settings. HBB has been shown to improve health worker skills and neonatal outcomes, but studies have documented problems with skills retention and little evidence of effectiveness at large scale in routine practice. This study examined the effect of complementing provider training with clinical mentorship and quality improvement as outlined in the second edition HBB materials. This “system-oriented” approach was implemented in all public health facilities (n = 172) in ten districts in Rwanda from 2015 to 2018. Methods A before-after mixed methods study assessed changes in provider skills and neonatal outcomes related to birth asphyxia. Mentee knowledge and skills were assessed with HBB objective structured clinical exam (OSCE) B pre and post training and during mentorship visits up to 1 year afterward. The study team extracted health outcome data across the entirety of intervention districts and conducted interviews to gather perspectives of providers and managers on the approach. Results Nearly 40 % (n = 772) of health workers in maternity units directly received mentorship. Of the mentees who received two or more visits (n = 456), 60 % demonstrated competence (received > 80% score on OSCE B) on the first mentorship visit, and 100% by the sixth. In a subset of 220 health workers followed for an average of 5 months after demonstrating competence, 98% maintained or improved their score. Three of the tracked neonatal health outcomes improved across the ten districts and the fourth just missed statistical significance: neonatal admissions due to asphyxia (37% reduction); fresh stillbirths (27% reduction); neonatal deaths due to asphyxia (13% reduction); and death within 30 min of birth (19% reduction, p = 0.06). Health workers expressed satisfaction with the clinical mentorship approach, noting improvements in confidence, patient flow within the maternity, and data use for decision-making. Conclusions Framing management of birth asphyxia within a larger quality improvement approach appears to contribute to success at scale. Clinical mentorship emerged as a critical element. The specific effect of individual components of the approach on provider skills and health outcomes requires further investigation.