Inter-Rater Reliability for the Amputation Endpoint in the National Vascular Quality Initiative.

Background: National quality reporting efforts after revascularization for peripheral artery disease (PAD) are ongoing. Validation of endpoints are necessary in national quality registries.
Objectives: This study sought to examine the interrater reliability for the endpoint of major amputation at 1 year in the Vascular Quality Initiative (VQI) registry and the Medicare-linked Vascular Quality Initiative registry (VQI-VISION) against electronic health record (EHR) review.
Methods: Surgical or endovascular revascularization procedures between January 1, 2010, and December 31, 2017, in the VQI registry and VQI-VISION for 2 academic health systems were queried. Major amputation data were abstracted by trained data collectors for the VQI and derived from Current Procedural Terminology codes for VQI-VISION. Cases underwent protocolized adjudication for the endpoint of major amputation by EHR review. Paired tests were used to evaluate the sensitivity and specificity. Spearman's ρ and Cohen's κ were used to evaluate interrater reliability.
Results: Amputation endpoints for 1,936 revascularizations were examined. Compared with major amputation data in EHR review, the sensitivity for the VQI registry was 35.9% and the specificity was 99.4% (ρ = 0.53; κ = 0.48). For VQI-VISION, sensitivity was 67.7% and specificity was 98.9% (ρ = 0.75; κ = 0.74). For any amputation in VQI data, sensitivity was 35.3% and specificity was 99.3% (ρ = 0.53; κ = 0.46), and for VQI-VISION, they were 71.6% and 97.7%, respectively (ρ = 0.75; κ = 0.74).
Conclusions: Almost two-thirds of the amputations in the VQI registry and one-third of amputations in VQI-VISION were missing at 1 year compared against adjudicated EHR review. In preparing for national reporting systems for major amputation tracking, data collection system reform is needed.
Competing Interests: Funding Support and Author Disclosures The VISION registry was supported by a Food and Drug Administration grant (U01FD006936 [to Dr Goodney]). Drs Smolderen and Mena are supported by National Institutes of Health grants 1R21AT012430-01 and 1R01HL163640-01A1. Dr Goodney is supported by a research grant from the American Heart Association (SRFN #18SFRN33900147). Dr Smolderen has received unrestricted research grants from Philips, Abbott, Merck, and Johnson & Johnson; and has served as a consultant for Optum Labs and Abbott, Happify, and Haemonetics. Dr Mena-Hurtado has served as a consultant for Abbott Vascular, Cook, Medtronic, Cardinal Health, and Optum Labs. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
(Copyright © 2024 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.)