Search

In the past, the lack of data on people with disabilities living in low- or middle-income countries (LMICs) has contributed to the invisibility of disability as a development priority. This is beginning to be addressed. While the Millennium Development Goals (MDGs) did not specifically mention disability, it is increasingly being recognised that the new post-MDG development agenda will be impossible to achieve without inclusion of people with disabilities. This systematic review examines the current evidence on the effectiveness of different interventions to improve the labour market participation of adults with disabilities in LMICs. The review scope covered a wide range of intervention strategies, populations, settings, and evaluation designs. 14 eligible impact evaluations published across the 20-year period 1992-2012 were identified. The type of evaluation design was quasi-experimental design with concurrent comparison group (five studies) or single-group pre-test/post-test (nine studies). Two studies used statistical methods to adjust for potentially confounding effects. Studies were conducted in a limited range of LMICs (five countries in Asia, three in Africa, and one in Latin America: three were low-income economies (Bangladesh, Kenya, and Zimbabwe), four were lower-middle income (India, Nigeria, Philippines, and Vietnam), and two were upper-middle income (Brazil and China). Populations with all impairment types were represented in the impact assessments, although most were focused on persons with physical disabilities. In all 14 studies the direction of effect was positive for the outcome variables measured. An appendix contains additional tables. [Sightsavers International provided additional funding for this report.]

BACKGROUND \ud The increase of low-income, low-skilled youth in the labour market, particularly in developing countries, is a major concern internationally. In some regions of the world, young people are nearly three times as likely as adults to be unemployed. They are also more likely to work in the informal labour market than adults, in low quality jobs that offer limited socio-economic security, training opportunities, and working conditions. This enormous unlocked potential represents a substantial loss of opportunity for both individuals and society. With increasing emphasis being given to work- and skills-based solutions to economic competition and poverty in the developing world, comes a renewed focus on technical and vocational education and training (TVET) as a means to expand opportunities for marginalised youth. Although several reviews have attempted to summarise the existing research in this area, there are a number of limitations to these reviews. There is a need to systematically examine the evidence base to provide a picture of the types of TVET interventions being used to raise employment, to identify those that are effective and ineffective, and to identify areas in which more research needs to be conducted. OBJECTIVES The main objective of this systematic review was to summarise the available evidence on the effects of TVET interventions for young people in developing countries to inform policy, practice, and research. The questions guiding this study were: \ud - What are the effects of different models of technical and vocational education and training (TVET) interventions on the employment and employability outcomes of young people, aged 15-24 years, in low- and middle-income countries? \ud - What do the findings suggest about moderating effects?\ud \ud SEARCH STRATEGY \ud A systematic and comprehensive search was used to locate both published and unpublished studies. A wide range of major bibliographic databases were electronically searched, along with specialist and grey literature databases, and websites of relevant organisations. Reference lists of previous reviews and included studies were examined. In addition, we conducted forward citation checking exercises and attempted contact with authors and other relevant stakeholders. \ud \ud SELECTION CRITERIA \ud Studies eligible for inclusion in the review were required to meet several eligibility criteria. First, studies must have evaluated a TVET intervention. Second, studies must have investigated outcomes for young people aged 15-24 years. Third, the geographical location of the studies must have been a low-or middle-income country. Fourth, studies must have utilised an experimental or quasi-experimental research design, including random assignment, quasi-random assignment (and groups generated were shown to be equivalent, or there was sufficient information to permit calculation of pre-treatment group equivalence), non-random assignment with matching, or non-random assignment with statistical controls. Fifth, studies must have reported at least one eligible outcome variable measuring employment (e.g., gaining paid employment) or employability (e.g., changing attitudes to work, or gaining job search skills). Finally, the date of publication or reporting of the study must have been between 2000 and 2011. No language restrictions were applied. \ud \ud DATA COLLECTION AND ANALYSIS \ud The electronic literature search yielded a total of 8072 potentially relevant reports, 145 of which were retrieved for full-text screening and nine were judged relevant. Handsearching identified a further 46 eligible reports. A total of 30 studies, reported in 55 publications, met the eligibility criteria. However, due to resources limitations, four of the eight eligible Spanish language papers we identified were not included in the review. Of the 26 studies included in the review, 3 utilised a randomised controlled trial (RCT) design, and 23 utilised a quasi-experimental design. The studies were coded independently by pairs of reviewers using a structured coding tool. Descriptive analysis was undertaken to examine and describe data related to the characteristics of the included studies and interventions. Ten of the 26 studies had data that allowed calculation of effects sizes. The findings from these 10 studies were statistically combined using meta-analytic techniques. The effect sizes were calculated using the standardised mean difference, corrected for small sample bias (i.e., Hedges' g). Analysis of the mean effect size, the heterogeneity of effect sizes, and the relationship between effect size and characteristics of the studies, participants and interventions was conducted. RESULTS The 26 included studies assessed the effectiveness of 20 different TVET interventions from various countries in Latin America, the Caribbean, Europe, East Asia, South Asia and Sub-Saharan Africa. Publication dates ranged between 2001 and 2011. Study settings included ten upper-middle income countries (Argentina, Bosnia and Herzegovina, Brazil, Chile, China, Colombia, Dominican Republic, Latvia, Mexico, Panama and Peru); two lower-middle income countries (India and Bhutan); and one low-income country (Kenya). The following summary of evidence focuses on the results of the statistical analyses of 10 studies included in the review. Employment - The overall mean effect of TVET on paid employment was positive and significant; however, significant heterogeneity was observed (Q = 23.8; df = 7; p = 0.00124; I2 = 70.6%; tau2 = 0.0153). \ud - Four variables were tested for moderating effects.\ud - Evidence of a statistically significant relationship between study quality and effect size was observed (Qb = 6.49; p = 0.0108). It is reasonable, therefore, to conclude that the overall mean effect may be inflated and that our conclusions about treatment effect on paid employment should be based only on those studies rated medium quality (g=0.06; 95% CI [-0.01, 0.12]). \ud - No significant differences in mean effects were observed between studies according to type of TVET intervention (Qb = 1.43; p = 0.231), length of follow-up period (Qb = 0.273; p = 0.601), or gender (Qb = 2.1; p = 0.147). \ud Formal employment\ud The overall mean effect of TVET on formal employment was positive and significant; however, significant heterogeneity was observed (Q = 11.1; df = 4; p = 0.0256; I2 = 63.9%; tau2 = 0.0131). \ud - One variable was tested for moderating effects. \ud - Evidence of a statistically significant relationship between study quality and effect size was observed (Qb = 10.6; p = 0.00116). It is reasonable, therefore, to conclude that the overall mean effect may be inflated and that our conclusions about treatment effect on formal employment should be based only on those studies rated medium quality (g=0.12; 95% CI [0.05, 0.19]). Monthly earnings \ud - The overall mean effect of TVET on earnings was positive and significant; however, significant heterogeneity was observed (Q = 25.5; df = 8; p = 0.00128; I2 = 68.6%; tau2 = 0.00815). \ud - Four variables were tested for moderating effects. \ud - No evidence of a statistically significant relationship between study quality and effect size was observed (Qb = 0.204; p = 0.652. It is reasonable, therefore, to conclude that the overall mean effect is not inflated and that our conclusions about treatment effect on monthly earnings should be based on all studies in the analysis (g=0.127; 95% CI [0.043, 0.21]). \ud - No statistically significant differences in mean effects were observed between studies according to type of TVET intervention (Qb = 0.397; p = 0.529), length of follow-up period (Qb = 0.186; p = 0.666), or gender (Qb = 1.26; p = 0.262). Self-employment earnings \ud - The overall mean effect of TVET on self-employment earnings was negative and non-significant (g=-0.025, 95% CI [-0.11, 0.061]). No significant heterogeneity was observed (Q = 0.206; df = 1; p = 0.65; I2 = 0%; tau2 = 0). This analysis was based on two medium quality studies. \ud - One variable was tested for moderating effects. \ud - No significant differences in mean effects were observed between studies according to gender (Qb = 1.27; p = 0.259). \ud Weekly hours worked \ud - The overall mean effect of TVET on number of weekly hours worked was positive but non-significant. No significant heterogeneity was observed (Q = 1.8; df = 5; p = 0.876; I2 = 0%; tau2 = 0). \ud - Four variables were tested for moderating effects. \ud - No evidence of a statistically significant relationship between study quality and effect size was observed (Qb = 1.41; p = 0.234). It is reasonable, therefore, to conclude that the overall mean effect is not inflated and that our conclusions about treatment effect on weekly hours should be based on all studies in the analysis (g=0.043; 95% CI [-0.017, 0.104]). \ud - Statistically significant differences in mean effects were observed between studies according to gender (Qb = 10.1; p = 0.00151). Treatment effects for female youth were positive, g=0.16 (95% CI [0.04, 0.28]), while those for male youth were negative, g=-0.09 (95% CI [-0.2, 0.01]). \ud - No significant differences in mean effects were observed between studies according to type of TVET intervention (Qb = 0.0677; p = 0.795), or length of follow-up period (Qb = 0.109; p = 0.741). \ud \ud AUTHORS' CONCLUSIONS \ud The studies included in this systematic review represent the best empirical evidence currently available for the impact of TVET on youth employment outcomes. As the review improves upon prior work by statistically synthesising TVET intervention research, its findings strengthen the evidence base on which current policies and practices can draw. That being said, interpreting the evidence and drawing out the implications for policy and practice is nonetheless challenging. Although this review provides some evidence of the causal impact of TVET on certain labour market outcomes, several limitations of both the included studies and the review itself mean that drawing strong inferences from the results of the analyses is not recommended and caution should be used when applying the findings of the review. A number of additional points are worth emphasising. First, attempts to explain the observed heterogeneity in overall mean effects suggest that methods matter. The low quality studies have consistently larger mean effects than the medium quality studies. In addition, for paid employment, and formal employment, statistically significant differences in mean effects were observed between studies according to study quality, suggesting that the overall mean is inflated and that the treatment effects should be based on the medium quality studies only. Second, effects are generally small and difficult to detect. The mean effects for paid employment (medium quality studies only), self-employment earnings, and working hours are relatively close to zero, and statistically insignificant. The mean effects for formal employment (medium quality studies only) and monthly earnings, although larger, are still relatively small, but they are statistically significant. Third, due to an insufficient number of studies reporting relevant data, only some of the variables for which moderator analyses had been planned a priori could be performed. Of the participant and intervention characteristics that were tested, only one demonstrated a significant relationship with treatment effect.. For weekly hours, statistically significant differences in mean treatment effects were observed between studies according to gender. It would be premature to conclude, however, that there are not in fact real differences between young men and women for other labour market outcomes, or between different types of TVET interventions, or that treatment effects do not diminish over time. We may not have had adequate statistical power to detect moderating effects of the variables tested in this review. There may be other moderating variables that could account for the differences in effects between studies that we were unable to test. In summary, the existing evidence shows that TVET interventions have some promise. Overall, interventions included in this review were found to demonstrate a small, positive effect on all but one of the employment outcomes measured, with the strength of the evidence being stronger for formal employment and monthly earnings than for the other outcomes measured. Furthermore, TVET appears to increase the number of hours worked in paid employment by young women but not young men. Thus, it is both important and worthwhile to continue to invest in TVET provision for youth in developing countries. Although, statistically, the overall effects of TVET may be small, even a small increase in the rate of paid employment, for example, could translate into large numbers of young people entering the labour market, where programmes are delivered nationally. A recommendation cannot be made either for, or against, any one type of TVET included in the review. So, in the absence of evidence in support of a particular, and possibly expensive, intervention, opting for the cheapest and/or most culturally acceptable models may be the best approach. At the same time, because the effects observed in this review are generally small and were difficult to detect, it is of some importance that future programmes are evaluated rigorously and that the different stakeholders involved think carefully about how to improve programmes to create larger effects on the outcomes. To build the evidence base further, many more of the TVET interventions currently in existence in developing countries need to be rigorously evaluated, and the results reported and disseminated efficiently. There is a clear need for additional research in this area. The methodological inconsistencies and weaknesses of the current evidence base, and specific knowledge gaps, suggest a number of future research priorities. These include: (a) evaluating all types of TVET; (b) testing the effects of different intervention components, and analysing all other relevant variables that may influence the effect; (c) measuring all key intermediate outcomes, long-term outcomes, and net outcomes; (d) improving reporting (e.g., description of interventions and outcome measures, data needed to calculate effect sizes, information needed for risk of bias judgments and study replication); and (e) evaluating the application of quasi-experimental techniques. Acting on these will also require the various stakeholders engaged in TVET research taking a critical look at the barriers affecting research production and dissemination.