Predictive validity of a quality measure for intensive substance use disorder treatment

Background Measures of substance use disorder (SUD) treatment quality are essential tools for performance improvement. The Veterans Health Administration (VHA) developed a measure of access to and engagement in intensive outpatient programs (IOPs) for SUD. However, predictive validity, or associations between this measure and treatment outcomes, has not been examined. Methods Data on veterans with SUD came from 3 samples: the Outcomes Monitoring Project ( N = 5436), a national evaluation of VHA mental health services ( N = 339,887), and patients receiving detoxification services ( N = 23,572). Propensity score–weighted mixed-effects regressions modeled associations between receiving at least 1 week of IOP treatment and patient outcomes, controlling for facility-level performance and a random effect for facility. Results Propensity score weighting reduced or eliminated observable baseline differences between patient groups. Patients who accessed IOPs versus those who did not reported significantly reduced alcohol- and drug-related symptom severity, with significantly fewer past-month days drinking alcohol ( b = 1.83, P < .001) and fewer past-month days intoxicated ( b = 1.55, P < .001). Patients who received IOP after detoxification services had higher 6-month utilization of SUD outpatient visits ( b = 2.09, P < .001), more subsequent detoxification episodes ( b = 0.25, P < .001), and lower odds of 2-year mortality (odds ratio [OR] = 0.68, 95% confidence interval [CI]: 0.61–0.75; P < .001). Conclusions Receiving at least 1 week of SUD treatment in an IOP was associated with higher follow-up utilization, improved health outcomes, and reduced mortality. These associations lend support to the predictive validity of VHA's IOP quality measure. Future research should focus on measure feasibility and validity outside of VHA, and whether predictive validity is maintained once this quality measure is tied to performance incentives.