Multi-task Learning Based on Multi-type Dataset for Retinal Abnormality Detection

The number of people suffering from ophthalmic diseases is increasing with the population aging. Many studies have been proposed to automatically identify diseases to reduce the risks of further retinal damage. However, most of existing methods mainly used a single type of dataset to solve the specific medical task, which is not clinically practical in the realworld scenarios. In this paper, we propose a multi-task deep learning network based on multi-types datasets to automatically recognise different ophthalmic diseases. Specifically, we first collect a multi-label dataset from the retinal fundus images and related diagnostic reports. Then, we propose a feature-fusion network to extract image and semantic retinal information from multi-types datasets. Finally, a multi-stream models is designed to integrate different specific features and realize the multiple disease detection. In this way, multi-types datasets based features are fully extracted in a multi-task learning manner. Experiments on our real-world dataset show that our proposed network significantly improve the classification performance of the model for ophthalmic diseases.