VET Program Completion Rates: An Evaluation of the Current Method. Occasional Paper

This work asks one simple question: "how reliable is the method used by the National Centre for Vocational Education Research (NCVER) to estimate projected rates of VET program completion?" In other words, how well do early projections align with actual completion rates some years later? Completion rates are simple to calculate with a cohort of students who start together in a very short program with a defined end date. The context in vocational education and training (VET) is, however, far more complex. Program lengths vary and may span several years, students commence at different times and many study part-time. Waiting for all students to complete or "drop out" of their training before calculating an actual completion rate gives a reliable answer, but is somewhat impractical. This paper summarises the key findings from a technical review of the validity of the method long used by NCVER in estimating projected completion rates for government-funded VET programs. This analysis required the interrogation of large longitudinal data sets with tens of millions of enrolments over multiple years. The outcomes are revealing because of ever-high interest in completion rates as measures of the efficiency and effectiveness of the VET sector. Key findings include the following: (1) The method long used by NCVER for estimating VET program completion rates using data from the National VET Provider Collection is shown to be reliable and aligns well with actual rates of completion for historical estimates; (2) Given that it takes a number of years for actual rates of completion to stabilise, the method is well suited for inclusion as part of any method of assessing completion rates, where the projected completion rate method is used to estimate rates for the most recent years and actual rates used for prior years; (3) The technical review has also shown that the current predictive method can be improved by defining a program's commencing year as the year it first appears in the National VET Provider Collection rather than using the commencing flag variable; and (4) It is anticipated that the incorporation of unique student identifiers into any preferred methodology, and its extension to total VET activity, can be phased in from the collection of 2017 training activity. "The Current Method for Projecting Rates of Completion: A Working Example" is appended.