Search

Across the world the Gender and Entrepreneurship Together (GET Ahead) training originally developed by the International Labour Organization has been implemented to improve business outcomes and enhance women’s empowerment. This randomized controlled trial is the first rigorous attempt to examine the impact of the GET Ahead training on women’s empowerment. We focus on the impact of offering this training to female microfinance borrowers of TYM, the largest microfinance organization in North Vietnam. A major contribution of this study is that it focuses on different dimensions of women’s empowerment: (1) personal empowerment, measured by control beliefs, and (2) relational empowerment, measured by relational friction and intra-household decision making power. This study also stands out in that we explicitly study whether involving husbands affects the impact of the training. We find that the GET Ahead training improves women’s empowerment on all three aspects: increased control beliefs and intra-household decision making power (only on larger expenditures), and decreased relational friction. However, the results on relational frictions should be taken with care due to possible underreporting. Moreover, in general, we find no additional impacts of inviting husbands to the training. Finally, our results suggest that it takes some time before the training starts to improve women’s empowerment. We observe no short-term but only mid-term effects from before the training to 12 months after the training.

Development is a multi-faceted process; achieving development goals thus requires a multi-sectoral approach. For over two decades, our research group of economists and nutritionists has designed and implemented randomized trials to assess the effectiveness of multisectoral programs in improving nutrition, food security, and other measures of well-being, largely at the request of developing country governments, development partners, and non-governmental organizations. Our approach addresses three perceived pitfalls of RCTs: the “black box” nature of RCTs, limited external validity, and challenges in translation of results to impacts at scale. We address these concerns by identifying and assessing programmatic pathways to impact with quantitative and qualitative methods; studying similar programs implemented by different organizations across various settings; and working closely with implementing partners in the design, research, and dissemination processes to inform adaptation and scale-up of programs and policies.

Highlights • It is unclear that what delivery model of early childhood interventions may be most effective in developing countries. • We evaluate the effects of a center-based parenting intervention on early development and parenting practices in rural China. • We compare the effects to a home-based intervention using the same curriculum and public service system in the same region. • We find the home-visiting intervention was more effective than the center-based parenting intervention on child outcomes. • The difference in effects appears to be driven by different patterns of compliance., We present the results of a cluster-randomized controlled trial that evaluates the effects of a free, center-based parenting intervention on early cognitive development and parenting practices in 100 rural villages in China. We then compare these effects to a previous trial of a home-based intervention conducted in the same region, using the same parenting curriculum and public service system, accounting for potential differences between the studies. We find that the center-based intervention did not have a significant impact on child development outcomes, but did lead to increases in the material investments, time investments, and parenting skills of caregivers. The average impact of the center-based intervention on child skills and investments in children was significantly smaller than the home-visiting intervention. Analysis of the possible mechanisms suggests that the difference in effects was driven primarily by different patterns of selection into program participation.

Highlights • Using an RCT, we assess the impacts of agriculture, nutrition, and gender interventions on women’s empowerment in Bangladesh. • Single or bundled trainings on agriculture, nutrition, and gender were provided to husbands and wives jointly. • All interventions improved women’s empowerment without disempowering men. • Nutrition trainings improved men’s gender attitudes, particularly on women’s responsibilities around cooking and childcare. • The role of engaging men and women jointly in interventions is a promising area for future research., The importance of women’s roles for nutrition-sensitive agricultural projects is increasingly recognized, yet little is known about whether such projects improve women’s empowerment and gender equality. We study the Agriculture, Nutrition, and Gender Linkages (ANGeL) pilot project, which was implemented as a cluster-randomized controlled trial by the Government of Bangladesh. The project’s treatment arms included agricultural training, nutrition behavior change communication (BCC), and gender sensitization trainings delivered to husbands and wives together – with these components combined additively, such that the impact of gender sensitization could be distinguished from that of agriculture and nutrition trainings. Empowerment was measured using the internationally-validated project-level Women’s Empowerment in Agriculture Index (pro-WEAI), and attitudes regarding gender roles were elicited from both men and women, to explore potentially gender-transformative impacts. Our study finds that ANGeL increased both women’s and men’s empowerment, raised the prevalence of households achieving gender parity, and led to small improvements in the gender attitudes of both women and men. We find significant increases in women’s empowerment scores and empowerment status from all treatment arms but with no significant differences across these. We find no evidence of unintended impacts on workloads and inconclusive evidence around impacts on intimate partner violence. Our results also suggest some potential benefits of bundling nutrition and gender components with an agricultural development intervention; however, many of these benefits seem to be driven by bundling nutrition with agriculture. While we cannot assess the extent to which including men and women within the same treatment arms contributed to our results, it is plausible that the positive impacts of all treatment arms on women’s empowerment outcomes may have arisen from implementation modalities that provided information to both husbands and wives when they were together. The role of engaging men and women jointly in interventions is a promising area for future research.

Although many studies have shown that sub-Saharan African countries could increase agricultural productivity and output through better technologies and improved crop varieties, the adoption of yield-enhancing technologies by smallholder farmers in the region still remains very low. In this paper, we use a cluster randomized controlled trial to examine whether improved extension services influence smallholder farmers’ decisions to try a newly introduced improved wheat variety in Ethiopia. The experiment involved demonstration trials and field days, and improved capacity of Development Agents in facilitation and communication skills. The empirical results revealed that farmers’ decisions to try the newly introduced wheat variety were highest in the villages where demonstration trials and field days, and improvements in the Development Agents’ facilitation and communication capacity were introduced simultaneously. We also found a positive effect of improved extension services on yields, and the use of chemical fertilizer, herbicides and pesticides, but only the use of herbicides and pesticides were statistically significant. Our findings provide some insights into the significance of revisiting the curriculums of the Technical and Vocational Education and Training, and Farmers’ Training Centers.

In response to mounting evidence on the dangers of irregular migration from Africa to Europe, the number of information campaigns which aim to raise awareness about the potential risks has rapidly increased. Governments, international organizations and civil society organizations implement a variety of campaigns to counter the spread of misinformation accelerated by smuggling and trafficking networks. The evidence on the effects of such information interventions on potential migrants remains limited and largely anecdotal. More generally, the role of risk perceptions in the decision-making process of potential irregular migrants is rarely explicitly tested, despite the fact that the concept of risk pervades conventional migration models, particularly in the field of economics. We address this gap by assessing the effects of a peer-to-peer information intervention on the perceptions, knowledge and intentions of potential migrants in Dakar, Senegal, using a randomized controlled trial design. The results show that – three months after the intervention – peer-to-peer information events increase potential migrants’ subjective information levels, raise risk awareness, and reduce intentions to migrate irregularly. We find no substantial effects on factual migration knowledge. We discuss how the effects may be driven by the trust and identification-enhancing nature of peer-to-peer communication.

We report on a randomized field experiment designed to relax credit and risk constraints for agricultural activities. We conducted a study in a drought-prone region in northern Ethiopia among poor smallholders who depended on rainfed agriculture and were members of the Productive Safety Net Programme (PSNP). Data were collected from over 1100 farmers in 32 rural villages over two years. We find that unconditional voucher transfers designated for the purchase of agricultural inputs significantly increased usage of seeds and fertilizers (a flypaper effect), raised the amount of farmland used (a complementary effect), and induced substitution of own effort by hiring casual labor (a local spillover effect). Subsidized rainfall insurance with reduced input vouchers produced weak average effects but greatly increased investments for farmers who were relatively more patient. We do not find heterogeneous effects by farmers’ risk attitudes, however, suggesting that the effects of insurance adoption were mainly determined by how farmers in the safety net made tradeoffs inter-temporally. Insurance demand dropped quickly with the reduction in subsidy and did not correlate with time or risk preference. Therefore, to improve cost-effectiveness, insurance programs should include procedures that help identify forward-looking farmers and encourage their adoption. While our results show that initial subsidies increase future insurance demand, the effect was small and thus initial subsidies would not be a cost-effective mechanism for financially sustainable insurance. Other complementary strategies on the design, promotion, and bundling techniques of insurance would be needed.

The use of Randomized Controlled Trials (RCTs) to evaluate development programs presents important challenges and opportunities for collaboration between academics and practitioners. Our team of experts measured the effectiveness of multi-stakeholder forums in 107 villages in Uganda over three years using an RCT approach. The focused technical demands of an RCT and the broad reality of practitioner worlds forced the team to tackle questions of: (a) how to manage the tensions of qualitative and quantitative data given the demand for evaluation procedures like power analysis estimation and hypothesis testing; (b) how to define measurable impact so it represents program realities and is amenable to the standardization required by an RCT; and (c) what compromises are required to maintain the spirit and practice of collaboration. We conclude that meaningful collaboration requires immense effort and compromise to leverage the significant benefits possible under this research paradigm.

Randomized control trials (RCTs) lie at the heart of the experimental approach to alleviating global poverty, stressing how the mechanics of household choice can be altered to improve welfare. Yet because households are not unified but rather have their own structural characteristics it cannot be assumed that the choice to participate in or the benefits from an intervention are equitably distributed within the household. Similarly, individual choice within and beyond households are constrained by the structure of social-property relations. Thus, choice cannot be isolated from the structures within which members of households live, and which act to constrain agency.

With minimal statistical or theoretical assumptions, randomized controlled trials (RCTs) provide a necessary input for poverty analysis: credibly estimated causal relationships. But complexities arise when moving from RCT research results to anti-poverty policy, with unintended consequences. RCT evidence by itself offers an incomplete prediction of the effects of policy, due to heterogenous effects, spillovers and general equilibrium changes, macroeconomic and welfare effects, political economy reactions, and implementation challenges, when programs are scaled. We suggest strategies for tightening the link between development research and anti-poverty policy, for example, by changing the practice of RCTs to be more ambitious about what is randomized, and to combine the analysis of experimental data with other rigorous methods that go beyond estimating treatment effects. We describe our efforts to encourage and coordinate this type of work via a new research initiative.

Ethical concerns aside, there is nothing inherently wrong with using randomized control trials for intellectual inquiry in development economics. A fundamental problem arises, however, in claiming that results from experimental and quasi-experimental methods are more credible than other sources of evidence for policy. Specifically, there is a contradiction between rejecting econometric assumptions required for identifying causal relationships using non-experimental data, and accepting assumptions required for extrapolating experimental results for policy. I explain this tension and its implications, then discuss recent efforts -- including the use of replication and machine learning methods -- to circumvent it. Such attempts remain inadequate, and assertions in the 2019 Nobel Award are therefore either premature or misplaced. Use of pluralistic approaches negates these sharp contradictions, but requires abandoning any special status for experimental methods.

Landscape-scale conservation programs are challenging to implement, and even more difficult to evaluate. Fundacion Natura Bolivia and associated researchers have spent the last decade undertaking a series of randomized control trials (RCTs) of an incentive-based conservation program in Bolivia. Large RCTs are complex, perhaps more so in conservation, as they require measurement of multiple kinds of outcomes operating on different timescales. We have learned that successful RCTs of conservation interventions require that program implementers demonstrate seven characteristics, namely that they are able and willing to: replicate a proven intervention at scale, define and measure outcomes, risk their reputation, have patience, access world-class technical research support, inculcate a tight researcher/practitioner collaboration and adapt the intervention based on evaluation results. Importantly, we have shown that large-scale robust RCT-based evaluations are possible in conservation. Learning how to use such evaluation tools is critical if conservation practitioners are to demonstrate attributable impact of their interventions.

Critics of randomized control trials (RCTs) in development economics argue that this methodology lends itself to ‘smaller’ questions with limited relevance to policy or economics. Using the seminal work of Miguel and Kremer (2004) on a school-based deworming intervention in Kenya as a case study, we argue that RCTs can spearhead policy change, serve as a laboratory to test economic theories and develop cutting-edge empirical methods, or do both. This does not happen in a vacuum, but through thoughtful design embedded in a broader research and policy agenda. Here, we describe a family of studies built on Miguel and Kremer (2004), shedding light on factors that supported the generation of evidence and insights far beyond the near-term RCT result. As in any piece of social sciences research, this descriptive evidence may not be externally valid in all settings. We nevertheless hope the lessons it offers will inspire others to examine these possibilities in their own research.

The new Nobel prize winners have expertly popularized randomized controlled trials (RCTs) as the “tool-of-choice” for empirical research. The award is a good opportunity to reflect on the role of RCTs in development-policy evaluation. Unbiasedness is the tool’s main virtue; transparency is another. Practitioners should also be aware of some limitations. First, an RCT assigns the treatment in a different way to most real-world policies, which use purposive selection; given heterogeneous impacts, one is evaluating a different intervention. Second, the tool may only be feasible for non-random subsets of both the relevant populations and the policy options, biasing assessments of overall development effectiveness. Third, given budget-constraints and a bias-variance trade-off, a non-RCT may allow a larger sample size, making its trials often closer to the truth. There is a continuing need for a broad range of research methods for addressing pressing knowledge gaps in fighting poverty.

This contribution reflects on lessons about research methods from GrOW, a large program on women's economic empowerment. GrOW encouraged multiple methods and inter-disciplinarity, and adopted experimental and quasi-experimental methods, survey data analysis, in-depth interviews, and PhotoVoice. We find that choices of and preferences for methods and measures do not lead to exclusion of complementary and other approaches – and this applied to the experimental studies as much as the others. Solution- and policy-oriented research requires combinations of methods, including demonstrating the ‘why’ as well as the ‘what’, and the capacity to do this well is not widespread.

Ten years ago, we offered reflections on the power and pitfalls of randomized controlled trials in development economics, arguing that the research community had lost its balance between theory, observational data and randomized experiments. We remain convinced of both the importance and the limits of RCTs for development economics research. But with another decade of RCTs under our collective belts, three issues now strike us as having become increasingly important. First, ethical risks still loom large. Second, increasing evidence that many interventions have highly heterogeneous impacts, places a premium on reintegrating ex ante theorizing with RCT methods to understand the heterogeneity. In some cases, heterogeneity may imply RCTs are less desirable than other research methods. Third, the increasing use of RCTs to study informational, behavioral, and other perceptions-mediated interventions creates an opportunity for non-classical measurement error problems that undercut the statistical power of seemingly well-designed studies in ways that remain underappreciated.

If there was ever an economic debate that randomized controlled trials could help resolve, it seemed to be the debate over the average economic and social impact of microcredit. When the first RCTs were published in 2015, they undermined beliefs in the potential to reduce mass poverty through microcredit, cutting through years of methodological debate. In retrospect, however, the studies reveal challenges in drawing inferences across RCTs. By design, the studies focus on marginal customers and marginal locations. As a result, the RCTs are most interesting and informative on their own terms and in their own idiosyncratic contexts. While it is tempting to interpret the results broadly, the studies were never designed to measure the average impact of microcredit. Ultimately, the RCTs shifted views on the possibilities for expanding microcredit and generated valuable insights, but they also showed that a diversity of methods—from RCTs that explore other margins to ethnography and financial diaries—is required to assess the sector’s overall contributions.

What do journals look for when vetting papers that use experimental designs? This piece provides an editor’s perspective on handling manuscripts that incorporate randomized approaches—and explains how to successfully present rigorous research based upon these methodologies. After considering bigger-picture issues like the underlying research questions of randomized controlled trials and their potential contributions, it discusses what information is important to include for adequately and transparently describing the studies themselves. Readers will learn how to prepare articles involving randomization, while gaining insight into the editorial process at World Development and other journals with similar standards.

It is important to test whether one intervention proves to be more successful than another at achieving the same goal, before deciding which one to introduce or scale up. Using randomised controlled trials (RCTs) can be useful when comparing efficacy, be that of medical treatments, or international development interventions. However, the research community should be careful not to let the RCT tail wag the implementation dog. A balance needs to be struck, between researchers primarily focusing on the research standards and data-gathering protocols on the one hand; and ensuring that the implementation of the interventions under study are carried out with integrity, on the other.

We present experimental evidence from a school choice program carried out in a 20-square kilometer, highly urbanized slum area known as Shahdara, which is situated in East Delhi, India. The lottery-based allocation of vouchers allows us to structure an impact evaluation as a randomized controlled trial. We conduct an Intent-to-Treat (ITT) analysis of the impact of the offer of a voucher as well as a Treatment-on-Treated (TOT) analysis of the impact of using a voucher, employing the lottery results as an instrumental variable. Four years after random assignment, we find large positive impacts of voucher use on student test scores in English (0.31σ, p

Summary. — There is little evidence showing whether health information transmitted via text messages can change health and educational outcomes. We conducted a randomized field experiment involving 900 primary students in rural China to study whether a health education campaign conducted via text message could affect caregiver knowledge or student outcomes. When caregivers received both weekly health messages and monthly quiz questions (testing retention of the information conveyed in the weekly messages), caregiver knowledge improved and students experienced gains in both health and academic performance. When caregivers received weekly health messages only, there was no impact on caregiver knowledge or student outcomes. 2014 Elsevier Ltd. All rights reserved.

It is common for health and nutrition interventions to target specific household members and for evaluations of their effects to focus exclusively on those members. However, if a targeted intervention changes a household's utility maximization problem or influences decision-making, households might respond to the intervention in unintended ways with the potential to affect the wellbeing of non-targeted members. Using panel data from a randomized controlled nutrition trial in Ghana, we evaluate household behavioral responses to the provision of small-quantity lipid-based nutrient supplements (SQ-LNS) to mothers and their infants to prevent undernutrition. We find that targeted supplementation with SQ-LNS had a positive effect on household expenditures on food, including some nutrient-rich food groups, as well as on non-food goods and services. We also find a positive impact on labor income, particularly among fathers. We then explore intrahousehold spillover effects on the nutritional status of non-targeted young children in the household. We find evidence that the targeted provision of SQ-LNS led to higher height-for-age z-scores among non-targeted children in the LNS group compared to the non-LNS group, though only among those with relatively taller mothers, which is an indicator of a child's growth potential. These findings support existing evidence and suggest that unintended behavioral responses and spillover are a real possibility in the context of nutrition interventions targeting nutritionally-vulnerable household members. Thoughtfully considering this possibility in the design, analyses, and evaluation of targeted nutrition interventions may provide a more complete picture of overall effects.