1. Benchmarking of deep learning algorithms for 3D instance segmentation of confocal image datasets
- Author
-
Guillaume Cerutti, Christophe Godin, Jan Traas, Yassin Refahi, Anuradha Kar, Petit M, Reproduction et développement des plantes (RDP), École normale supérieure de Lyon (ENS de Lyon)-Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-Université de Lyon-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE), Institut du Cerveau = Paris Brain Institute (ICM), Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Sorbonne Université (SU)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS), Simulation et Analyse de la morphogenèse in siliCo (MOSAIC), Inria Lyon, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Fractionnement des AgroRessources et Environnement (FARE), Université de Reims Champagne-Ardenne (URCA)-Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE), École normale supérieure - Lyon (ENS Lyon)-Université Claude Bernard Lyon 1 (UCBL), and ROSSI, Sabine
- Subjects
Computer science ,Confocal ,media_common.quotation_subject ,[INFO.INFO-IM] Computer Science [cs]/Medical Imaging ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Adaptability ,Image (mathematics) ,Cellular and Molecular Neuroscience ,Deep Learning ,Imaging, Three-Dimensional ,Market segmentation ,[INFO.INFO-IM]Computer Science [cs]/Medical Imaging ,Genetics ,Image Processing, Computer-Assisted ,Segmentation ,Molecular Biology ,Ecology, Evolution, Behavior and Systematics ,media_common ,Ecology ,business.industry ,Deep learning ,Pattern recognition ,Image segmentation ,Visualization ,Benchmarking ,Computational Theory and Mathematics ,Modeling and Simulation ,Artificial intelligence ,business ,Algorithms - Abstract
Segmenting three dimensional microscopy images is essential for understanding phenomena like morphogenesis, cell division, cellular growth and genetic expression patterns. Recently, deep learning (DL) pipelines have been developed which claim to provide high accuracy segmentation of cellular images and are increasingly considered as the state-of-the-art for image segmentation problems. However, it remains difficult to define their relative performances as the concurrent diversity and lack of uniform evaluation strategies makes it difficult to know how their results compare. In this paper, we first made an inventory of the available DL methods for 3 dimensional (3D) cell segmentation. We next implemented and quantitatively compared a number of representative DL pipelines, alongside a highly efficient non-DL method named MARS. The DL methods were trained on a common dataset of 3D cellular confocal microscopy images. Their segmentation accuracies were also tested in the presence of different image artifacts. A specific method for segmentation quality evaluation was adopted which isolates segmentation errors due to under/over segmentation. This is complemented with a 3D visualization strategy for interactive exploration of segmentation quality. Our analysis shows that the DL pipelines have different levels of accuracy. Two of them, which are end to end 3D and were originally designed for cell boundary detection, show high performance, and offer clear advantages in terms of adaptability to new data.Author summaryIn recent years a number of deep learning (DL) algorithms based on computational neural networks have been developed which claim to achieve high accuracy and automatic segmentation of 3D microscopy images. Although these algorithms have received considerable attention in the literature, it is difficult to evaluate their relative performances, while it remains unclear whether they really perform better than other, more classical segmentation methods.To clarify these issues, we performed a detailed, quantitative analysis of a number of representative DL pipelines for cell instance segmentation from 3D confocal microscopy image datasets. We developed a protocol for benchmarking the performances of such DL based segmentation pipelines using common training and test datasets, evaluation metrics and visualizations. Using this protocol, we evaluated and compared four different DL pipelines to identify their strengths and limitations. A high performance non-DL method was also included in the evaluation. We show that DL pipelines may show significant differences in their performances depending on their model architecture and pipeline components but overall show excellent adaptability to unseen data. We also show that our benchmarking protocol could be extended to a variety of segmentation pipelines and datasets.
- Published
- 2022