Search

Split-plot experimental data are often analyzed as if the data came from a completely randomized design. As is well known, ignoring the different levels of randomization and replication can lead to serious inferential errors. However, in some experiments, including many of the ocean global change experiments that motivated this research, variation between whole-plot experimental units may be small relative to variation between subplot units. Even though a factorial analysis will often perform poorly in general, in this special case it outperforms a split-plot analysis, providing narrower confidence intervals for treatment means and differences with coverage rates close to the desired level. The performance of the proposed model-averaged analysis was compared to a classical split-plot analysis via a simulation study, and its utility demonstrated for an ocean global change experiment examining growth and condition of a juvenile mussel species. In our simulation study, model-averaged confidence intervals for whole-plot treatment means or comparisons of means were up to 40% narrower than split-plot confidence intervals while maintaining close to nominal coverage rates. In our example experiment, we observed narrowing of up to 25%. We recommend model averaging as a preferred approach when variation between whole-plot experimental units is expected to be less than between subplot units, with a few caveats for studies with very few replicates. [ABSTRACT FROM AUTHOR]

Increasing coverage of evidence-based maternal, neonatal, child, reproductive health and nutrition (MNCRHN) programs in low- and middle-income countries has coincided with dramatic improvements in health despite variable quality of implementation. Comprehensive evaluation to inform program improvement requires standardized but adaptable tools, which the Real Accountability, Data Analysis for Results (RADAR) project has developed. To inform selection of tools and methods packages ('packages') to measure program quality of care (QoC), we documented experiences testing the packages, which were developed and adapted based on global and local expertise, and pre- and pilot-testing. We conducted cross-sectional studies in 2018–2019 on the quality of 1) integrated community case management, 2) counseling on maternal, infant, and young child feeding, 3) intrapartum care, and 4) family planning counseling in Mali, Mozambique, Tanzania, and Malawi. Herein we describe package performance and highlight experiences that inform their selection and use. Direct observation packages provided high-quality, immediately applicable results but they required specialized expertise, in-person collection, adequate patient volume, reasonable wait times, and unambiguously 'correct' provision of care. General satisfaction questions from exit interview packages produced unvaryingly positive responses despite variable observed quality of care. Variation increased when questions were more targeted, but findings on caregiver and client's recall of recommendations were more actionable. When interactive, clinical vignettes can capture knowledge of clinical care. But for conditions that can be simulated, like provision of family planning counseling, we could capture provider practice from simulated clients. Clinicians could more easily demonstrate tactile aspects of intrapartum care using observed structured clinical examinations, but this method required storage and transport of the required mannequins. Based on our findings we recommend ten questions upon which evaluators can base package selection. Findings from these packages inform programs and, in the context of comprehensive program evaluation enable us to link programs with impact. [ABSTRACT FROM AUTHOR]

Many countries have adopted integrated community case management (iCCM) to reduce mortality among children under five years from common childhood illnesses. The 2016–2020 Malian Red Cross iCCM program trained 441 Community Health Workers (CHWs) to treat malaria, pneumonia, diarrhea, and malnutrition for children under five years of age in six districts. Implementation strength and quality of care (QoC) were assessed through the program's supervision function, using the Malian Ministry of Health's system. This paper compares methods and results of program supervision data and an independent evaluation to assess the effectiveness of program implementation and supervision and inform program improvement. It also presents the benefits and limitations of each method. An independent QoC evaluation was conducted using tools developed by the Real Accountability: Data Analysis for Results (RADAR) project, hereafter referred to as the RADAR evaluation. RADAR evaluation data collected in July and August 2018 were compared with program supervision data collected mostly between May and December 2018. The RADAR evaluation provided detailed findings on correct assessment, classification, and treatment per illness, medication type, and dosage. Program supervision combined the findings for all illnesses, medication type, and dosage due to limitations in the data collection process. Six indicators were comparable between both methods. Findings were similar for temperature and mid-upper arm circumference measurements but diverged between program supervision and the RADAR evaluation, respectively, on correct classification for all illnesses (87.1% vs. 65.3%), correct treatment for all illnesses (69.5% vs. 39.8%), correct respiratory rate counting (88.5% vs. 54.7%), and administering the first dose by CHW (75.4% vs. 65.0%). Findings from the RADAR evaluation guided improvements in program supervision. A robust program supervision system can serve as a credible method to assess QoC. However, a rigorous independent QoC evaluation provides a valuable benchmark to gauge the effectiveness of the supervisory process. [ABSTRACT FROM AUTHOR]