Multi-institutional Assessment of Pathologist scoring HER2 Immunohistochemistry

The HercepTest was approved 20 + years ago as the companion diagnostic test for trastuzumab in HER2 amplified/overexpressing breast cancers. Subsequent HER2 immunohistochemistry (IHC) assays followed, including the now most common Ventana 4B5 assay. While this IHC assay has become the clinical standard, its reliability, reproducibility, and accuracy have largely been approved and accepted based on concordance between small numbers of pathologists without validation in a real-world setting. In this study, we evaluate the concordance and inter-rater reliability of scoring HER2 IHC in 170 breast cancer biopsies by 18 breast cancer-specialized pathologists from 15 institutions. We used the ONEST (Observers Needed to Evaluate Subjective Tests) method to determine the plateau of concordance and the minimum number of pathologists needed to estimate inter-rater agreement values for large numbers of raters, as seen in the real-world setting. We report substantial discordance within the intermediate categories (