1. Multimodal MALDI imaging mass spectrometry for improved diagnosis of melanoma.
- Author
-
Zhang W, Patterson NH, Verbeeck N, Moore JL, Ly A, Caprioli RM, De Moor B, Norris JL, and Claesen M
- Subjects
- Humans, Skin Neoplasms diagnosis, Skin Neoplasms pathology, Neural Networks, Computer, Deep Learning, Multimodal Imaging methods, Melanoma diagnosis, Melanoma pathology, Spectrometry, Mass, Matrix-Assisted Laser Desorption-Ionization methods
- Abstract
Imaging mass spectrometry (IMS) provides promising avenues to augment histopathological investigation with rich spatio-molecular information. We have previously developed a classification model to differentiate melanoma from nevi lesions based on IMS protein data, a task that is challenging solely by histopathologic evaluation. Most IMS-focused studies collect microscopy in tandem with IMS data, but this microscopy data is generally omitted in downstream data analysis. Microscopy, nevertheless, forms the basis for traditional histopathology and thus contains invaluable morphological information. In this work, we developed a multimodal classification pipeline that uses deep learning, in the form of a pre-trained artificial neural network, to extract the meaningful morphological features from histopathological images, and combine it with the IMS data. To test whether this deep learning-based classification strategy can improve on our previous results in classification of melanocytic neoplasia, we utilized MALDI IMS data with collected serial H&E stained sections for 331 patients, and compared this multimodal classification pipeline to classifiers using either exclusively microscopy or IMS data. The multimodal pipeline achieved the best performance, with ROC-AUCs of 0.968 vs. 0.938 vs. 0.931 for the multimodal, unimodal microscopy and unimodal IMS pipelines respectively. Due to the use of a pre-trained network to perform the morphological feature extraction, this pipeline does not require any training on large amounts of microscopy data. As such, this framework can be readily applied to improve classification performance in other experimental settings where microscopy data is acquired in tandem with IMS experiments., Competing Interests: JLM, NHP, SN, RMC, JLN, and JR disclose a financial interest in Fron- tier Diagnostics, LLC (FDx). FDx has issued and pending patent appli- cations in the US Patent Office that include part of the methods described in this paper. NV and MC, principals of Aspect Analytics NV, are paid consultants and provide services to FDx. This does not alter our adherence to PLOS ONE policies on sharing data and materials., (Copyright: © 2024 Zhang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)
- Published
- 2024
- Full Text
- View/download PDF