1. Convolutional Network With Twofold Feature Augmentation for Diabetic Retinopathy Recognition From Multi-Modal Images
- Author
-
Sung-Ho Bae, Sungyoung Lee, Ki-Young Kim, Thuong Le-Tien, Thien Huynh-The, Seung-Young Yu, Cam-Hao Hua, and Jong In You
- Subjects
Fundus Oculi ,Computer science ,Feature vector ,Feature extraction ,02 engineering and technology ,Retina ,03 medical and health sciences ,Quadratic equation ,Health Information Management ,Robustness (computer science) ,Diabetes Mellitus ,0202 electrical engineering, electronic engineering, information engineering ,medicine ,Humans ,Electrical and Electronic Engineering ,030304 developmental biology ,0303 health sciences ,Diabetic Retinopathy ,Receiver operating characteristic ,business.industry ,Deep learning ,Angiography ,Pattern recognition ,Diabetic retinopathy ,medicine.disease ,Computer Science Applications ,Modal ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,Tomography, Optical Coherence ,Biotechnology - Abstract
Objective: With the scenario of limited labeled dataset, this paper introduces a deep learning-based approach that leverages Diabetic Retinopathy (DR) severity recognition performance using fundus images combined with wide-field swept-source optical coherence tomography angiography (SS-OCTA). Methods: The proposed architecture comprises a backbone convolutional network associated with a Twofold Feature Augmentation mechanism, namely TFA-Net. The former includes multiple convolution blocks extracting representational features at various scales. The latter is constructed in a two-stage manner, i.e., the utilization of weight-sharing convolution kernels and the deployment of a Reverse Cross-Attention (RCA) stream. Results: The proposed model achieves a Quadratic Weighted Kappa rate of 90.2% on the small-sized internal KHUMC dataset. The robustness of the RCA stream is also evaluated by the single-modal Messidor dataset, of which the obtained mean Accuracy (94.8%) and Area Under Receiver Operating Characteristic (99.4%) outperform those of the state-of-the-arts significantly. Conclusion: Utilizing a network strongly regularized at feature space to learn the amalgamation of different modalities is of proven effectiveness. Thanks to the widespread availability of multi-modal retinal imaging for each diabetes patient nowadays, such approach can reduce the heavy reliance on large quantity of labeled visual data. Significance: Our TFA-Net is able to coordinate hybrid information of fundus photos and wide-field SS-OCTA for exhaustively exploiting DR-oriented biomarkers. Moreover, the embedded feature-wise augmentation scheme can enrich generalization ability efficiently despite learning from small-scale labeled data.
- Published
- 2021