Evaluating a novel neonatal-care assessment tool among trained delivery attendants in a resource-limited setting

Objective To validate a novel objective structured clinical examination (OSCE) tool for assessing neonatal care skills among delivery attendants trained as part of the Essential Care for Every Baby (ECEB) program and to assess ECEB training effectiveness. Methods Between August 1 and September 30, 2015, a cross-sectional study enrolled ECEB-trained healthcare providers who attended deliveries from the Brong Ahafo and Eastern regions of Ghana. Participants completed a previously developed 21-item OSCE tool that assessed neonatal-care competency. Participant performance was scored independently by regional trainers and national master trainers. The inter-rater scoring reliability was assessed using the Cohen kappa coefficient and performance was compared across participant characteristics. Results The study enrolled 57 trained delivery attendants from 12 district hospitals. Inter-rater agreement was perfect (kappa 1.00) or almost perfect (kappa 0.81–0.99) for nine OSCE items, substantial (kappa 0.61–0.80) or moderate (kappa 0.41–0.60) for 11 items, and fair (kappa 0.21–0.40) for one item. Differences in OSCE-item performance were recorded based on participants’ regions, facility type, age, and education level ( P Conclusions In a resource-limited setting, the OSCE tool demonstrated substantial reliability and ECEB-trained healthcare practitioners exhibited satisfactory performance. The OSCE tool could be useful in similar settings and could have potential for up-scaled use in assessing neonatal-management skills.