Back to Search
Start Over
Crowdsourcing image segmentation for deep learning: integrated platform for citizen science, paid microtask, and gamification.
- Source :
-
Biomedizinische Technik. Biomedical engineering [Biomed Tech (Berl)] 2023 Dec 26; Vol. 69 (3), pp. 293-305. Date of Electronic Publication: 2023 Dec 26 (Print Publication: 2024). - Publication Year :
- 2023
-
Abstract
- Objectives: Segmentation is crucial in medical imaging. Deep learning based on convolutional neural networks showed promising results. However, the absence of large-scale datasets and a high degree of inter- and intra-observer variations pose a bottleneck. Crowdsourcing might be an alternative, as many non-experts provide references. We aim to compare different types of crowdsourcing for medical image segmentation.<br />Methods: We develop a crowdsourcing platform that integrates citizen science (incentive: participating in the research), paid microtask (incentive: financial reward), and gamification (incentive: entertainment). For evaluation, we choose the use case of sclera segmentation in fundus images as a proof-of-concept and analyze the accuracy of crowdsourced masks and the generalization of learning models trained with crowdsourced masks.<br />Results: The developed platform is suited for the different types of crowdsourcing and offers an easy and intuitive way to implement crowdsourcing studies. Regarding the proof-of-concept study, citizen science, paid microtask, and gamification yield a median F-score of 82.2, 69.4, and 69.3 % compared to expert-labeled ground truth, respectively. Generating consensus masks improves the gamification masks (78.3 %). Despite the small training data (50 images), deep learning reaches median F-scores of 80.0, 73.5, and 76.5 % for citizen science, paid microtask, and gamification, respectively, indicating sufficient generalizability.<br />Conclusions: As the platform has proven useful, we aim to make it available as open-source software for other researchers.<br /> (© 2023 the author(s), published by De Gruyter, Berlin/Boston.)
Details
- Language :
- English
- ISSN :
- 1862-278X
- Volume :
- 69
- Issue :
- 3
- Database :
- MEDLINE
- Journal :
- Biomedizinische Technik. Biomedical engineering
- Publication Type :
- Academic Journal
- Accession number :
- 38143326
- Full Text :
- https://doi.org/10.1515/bmt-2023-0148