Deep Learning-Based Multi-Class Classification of Breast Digital Pathology Images

Weiming Mi,1,2,* Junjie Li,3,* Yucheng Guo,4,* Xinyu Ren,3 Zhiyong Liang,3 Tao Zhang,1,2 Hao Zou4,5 1Department of Automation, School of Information Science and Technology, Tsinghua University, Beijing, Peoples Republic of China; 2Beijing National Research Center for Information Science and Technology, Beijing, Peoples Republic of China; 3Molecular Pathology Research Center, Department of Pathology, Peking Union Medical College Hospital (PUMCH), Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing, Peoples Republic of China; 4Tsimage Medical Technology, Yantian Modern Industry Service Center, Shenzhen, Peoples Republic of China; 5Center for Intelligent Medical Imaging & Health, Research Institute of Tsinghua University in Shenzhen, Shenzhen, Peoples Republic of China*These authors contributed equally to this workCorrespondence: Zhiyong LiangMolecular Pathology Research Center, Department of Pathology, Peking Union Medical College Hospital (PUMCH), Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing, 100730, People’s Republic of ChinaEmail liangzhiyong1220@yahoo.comTao ZhangDepartment of Automation, School of Information Science and Technology, Tsinghua University, Beijing, 100084, People’s Republic of ChinaEmail taozhang@tsinghua.edu.cnIntroduction: Breast cancer, one of the most common health threats to females worldwide, has always been a crucial topic in the medical field. With the rapid development of digital pathology, many scholars have used AI-based systems to classify breast cancer pathological images. However, most existing studies only stayed on the binary classification of breast lesions (normal vs tumor or benign vs malignant), far from meeting the clinical demand. Therefore, we established a multi-class classification system of breast digital pathology images based on AI, which is more clinically practical than the binary classification system.Methods: In this paper, we adopted a two-stage architecture based on deep learning method and machine learning method for the multi-class classification (normal tissue, benign lesion, ductal carcinoma in situ, and invasive carcinoma) of breast digital pathological images.Results: The proposed approach achieved an overall accuracy of 86.67% at patch-level. At WSI-level, the overall accuracies of our classification system were 88.16% on validation data and 90.43% on test data. Additionally, we used two public datasets, the BreakHis and BACH, for independent verification. The accuracies our model obtained on these two datasets were comparable to related publications. Furthermore, our model could achieve accuracies of 85.19% on multi-classification and 96.30% on binary classification (non-malignant vs malignant) using pathology images of frozen sections, which was proven to have good generalizability. Then, we used t-SNE for visualization of patch classification efficiency. Finally, we analyzed morphological characteristics of patches learned by the model.Conclusion: The proposed two-stage model could be effectively applied to the multi-class classification task of breast pathology images and could be a very useful tool for assisting pathologists in diagnosing breast cancer.Keywords: image analysis, breast cancer, digital pathology images, deep learning, multi-class classification