1. Deep Multi-Instance Learning Using Multi-Modal Data for Diagnosis of Lymphocytosis
- Author
-
Eugenie Maurin, Béatrice Grange, Evangelia I. Zacharaki, Nikos Paragios, Laurent Jallades, Pierre Sujobert, Maria Vakalopoulou, Mihir Sahasrabudhe, Mathématiques et Informatique pour la Complexité et les Systèmes (MICS), CentraleSupélec-Université Paris-Saclay, OPtimisation Imagerie et Santé (OPIS), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre de vision numérique (CVN), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-CentraleSupélec-Université Paris-Saclay, Faculté de Médecine Lyon-Sud, FACULTE DE LYON, Cancer Research Center of Lyon, INSERM, Centre National de la Recherche Scientifique (CNRS), University of Patras [Patras], TheraPanacea [Paris], Sahasrabudhe, Mihir, Centre de Recherche en Cancérologie de Lyon (UNICANCER/CRCL), Centre Léon Bérard [Lyon]-Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-Université de Lyon-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS), and University of Patras
- Subjects
[INFO.INFO-AI] Computer Science [cs]/Artificial Intelligence [cs.AI] ,[INFO.INFO-TS] Computer Science [cs]/Signal and Image Processing ,Computer science ,Pipeline (computing) ,Feature extraction ,[INFO.INFO-IM] Computer Science [cs]/Medical Imaging ,Lymphocytosis ,02 engineering and technology ,Convolutional neural network ,multiple-instance learning ,mixture-of-experts ,[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI] ,Machine Learning ,03 medical and health sciences ,[INFO.INFO-CV] Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV] ,Text mining ,Health Information Management ,[INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] ,[INFO.INFO-TS]Computer Science [cs]/Signal and Image Processing ,convolutional neural networks ,0202 electrical engineering, electronic engineering, information engineering ,Feature (machine learning) ,Code (cryptography) ,[INFO.INFO-IM]Computer Science [cs]/Medical Imaging ,Humans ,Electrical and Electronic Engineering ,030304 developmental biology ,0303 health sciences ,business.industry ,Deep learning ,[INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV] ,Pattern recognition ,[INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG] ,3. Good health ,Computer Science Applications ,[INFO.INFO-TI] Computer Science [cs]/Image Processing [eess.IV] ,[INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV] ,Embedding ,multi-source data ,020201 artificial intelligence & image processing ,Neural Networks, Computer ,Artificial intelligence ,Feature aggregation ,business ,Biotechnology - Abstract
We investigate the use of recent advances in deep learning and propose an end-to-end trainable multi-instance convolutional neural network within a mixture-of-experts formulation that combines information from two types of data—images and clinical attributes—for the diagnosis of lymphocytosis. The convolutional network learns to extract meaningful features from images of blood cells using an embedding level approach and aggregates them. Moreover, the mixture-of-experts model combines information from these images as well as clinical attributes to form an end-to-end trainable pipeline for diagnosis of lymphocytosis. Our results demonstrate that even the convolutional network by itself is able to discover meaningful associations between the images and the diagnosis, indicating the presence of important unexploited information in the images. The mixture-of-experts formulation is shown to be more robust while maintaining performance via. a repeatability study to assess the effect of variability in data acquisition on the predictions. The proposed methods are compared with different methods from literature based both on conventional handcrafted features and machine learning, and on recent deep learning models based on attention mechanisms. Our method reports a balanced accuracy of $\text{85.41}\%$ and outperfroms the handcrafted feature-based and attention-based approaches as well that of biologists which scored $\text{79.44}\%$ , $\text{82.89}\%$ and $\text{77.07}\%$ respectively. These results give insights on the potentials of the applicability of the proposed method in clinical practice. Our code and datasets can be found at https://github.com/msahasrabudhe/lymphoMIL .
- Published
- 2020