Search

It is still debating if individualized dose should be applied for the emerging PSMA-targeted radionuclide therapy (RLT). A critical consideration in this debate is the necessity and feasibility of individual estimation of post-therapy dosimetry before the treatment. In this study, we aimed to prove the concept of individual dosimetry prediction based on pre-therapy imaging and laboratory measurements. Methods: 23 patients with metastatic castration-resistant prostate cancer (mCRPC) treated with 177Lu-PSMA-I&T RLT were included retrospectively. Included patients had available pre-therapeutic 68Ga-PSMA-HEBD-CC PET/CT and at least 3 planar and 1 SPECT/CT dosimetry imaging. Overall, 43 cycles of 177Lu-PSMA I&T RLT were applied. Organ-based standard uptake value (SUV) uptake was obtained from pretherapy PET/CT scans. Patient individual dosimetry was calculated for kidney, liver, spleen, and salivary glands using Hermes Hybrid Dosimetry 4.0 from the post-treatment 177Lu-PSMA I&T imaging studies. Machine learning methods were explored for individual dose prediction from PET images. The accuracy of these dose predictions was compared with the accuracy of population-based dosimetry estimates. Mean absolute percentage error was used to assess the prediction error of estimated dosimetry. Results: An optimal machine learning method achieved a dosimetry prediction error of 15.8 ± 13.2% for kidney, 29.6%±13.7% for liver, 23.8%±13.1% for salivary glands and 32.1 ± 31.4% for spleen. In contrast, the prediction based on literature population mean has significantly larger error (p Conclusion: The preliminary results confirmed the feasibility of individual estimation of post-therapy dosimetry before the RLT and its added value to empirical population-based estimation. The exploration of individual dose prediction may support the identification of the role of treatment planning for RLT.

Purpose Although treatment planning and individualized dose application for emerging prostate-specific membrane antigen (PSMA)-targeted radioligand therapy (RLT) are generally recommended, it is still difficult to implement in practice at the moment. In this study, we aimed to prove the concept of pretherapeutic prediction of dosimetry based on imaging and laboratory measurements before the RLT treatment. Methods Twenty-three patients with metastatic castration-resistant prostate cancer (mCRPC) treated with 177Lu-PSMA I&T RLT were included retrospectively. They had available pre-therapy 68 Ga-PSMA-HEBD-CC PET/CT and at least 3 planar and 1 SPECT/CT imaging for dosimetry. Overall, 43 cycles of 177Lu-PSMA I&T RLT were applied. Organ-based standard uptake values (SUVs) were obtained from pre-therapy PET/CT scans. Patient dosimetry was calculated for the kidney, liver, spleen, and salivary glands using Hermes Hybrid Dosimetry 4.0 from the planar and SPECT/CT images. Machine learning methods were explored for dose prediction from organ SUVs and laboratory measurements. The uncertainty of these dose predictions was compared with the population-based dosimetry estimates. Mean absolute percentage error (MAPE) was used to assess the prediction uncertainty of estimated dosimetry. Results An optimal machine learning method achieved a dosimetry prediction MAPE of 15.8 ± 13.2% for the kidney, 29.6% ± 13.7% for the liver, 23.8% ± 13.1% for the salivary glands, and 32.1 ± 31.4% for the spleen. In contrast, the prediction based on literature population mean has significantly larger MAPE (p Conclusion The preliminary results confirmed the feasibility of pretherapeutic estimation of treatment dosimetry and its added value to empirical population-based estimation. The exploration of dose prediction may support the implementation of treatment planning for RLT.

This study proposes an automated prostate cancer (PC) lesion characterization method based on the deep neural network to determine tumor burden on 68Ga-PSMA-11 PET/CT to potentially facilitate the optimization of PSMA-directed radionuclide therapy. We collected 68Ga-PSMA-11 PET/CT images from 193 patients with metastatic PC at three medical centers. For proof-of-concept, we focused on the detection of pelvis bone and lymph node lesions. A deep neural network (triple-combining 2.5D U-Net) was developed for the automated characterization of these lesions. The proposed method simultaneously extracts features from axial, coronal, and sagittal planes, which mimics the workflow of physicians and reduces computational and memory requirements. Among all the labeled lesions, the network achieved 99% precision, 99% recall, and an F1 score of 99% on bone lesion detection and 94%, precision 89% recall, and an F1 score of 92% on lymph node lesion detection. The segmentation accuracy is lower than the detection. The performance of the network was correlated with the amount of training data. We developed a deep neural network to characterize automatically the PC lesions on 68Ga-PSMA-11 PET/CT. The preliminary test within the pelvic area confirms the potential of deep learning methods. Increasing the amount of training data should further enhance the performance of the proposed method and may ultimately allow whole-body assessments.

Our aim was to introduce and validate qPSMA, a semiautomatic software package for whole-body tumor burden assessment in prostate cancer patients using 68Ga-prostate-specific membrane antigen (PSMA) 11 PET/CT. Methods: qPSMA reads hybrid PET/CT images in DICOM format. Its pipeline was written using Python and C++ languages. A bone mask based on CT and a normal-uptake mask including organs with physiologic 68Ga-PSMA11 uptake are automatically computed. An SUV threshold of 3 and a liver-based threshold are used to segment bone and soft-tissue lesions, respectively. Manual corrections can be applied using different tools. Multiple output parameters are computed, that is, PSMA ligand-positive tumor volume (PSMA-TV), PSMA ligand-positive total lesion (PSMA-TL), PSMA SUVmean, and PSMA SUVmax Twenty 68Ga-PSMA11 PET/CT data sets were used to validate and evaluate the performance characteristics of qPSMA. Four analyses were performed: validation of the semiautomatic algorithm for liver background activity determination, assessment of intra- and interobserver variability, validation of data from qPSMA by comparison with Syngo.via, and assessment of computational time and comparison of PSMA PET-derived parameters with serum prostate-specific antigen. Results: Automatic liver background calculation resulted in a mean relative difference of 0.74% (intraclass correlation coefficient [ICC], 0.996; 95%CI, 0.989;0.998) compared with METAVOL. Intra- and interobserver variability analyses showed high agreement (all ICCs > 0.990). Quantitative output parameters were compared for 68 lesions. Paired t testing showed no significant differences between the values obtained with the 2 software packages. The ICC estimates obtained for PSMA-TV, PSMA-TL, SUVmean, and SUVmax were 1.000 (95%CI, 1.000;1.000), 1.000 (95%CI, 1.000;1.000), 0.995 (95%CI, 0.992;0.997), and 0.999 (95%CI, 0.999;1.000), respectively. The first and second reads for intraobserver variability resulted in mean computational times of 13.63 min (range, 8.22-25.45 min) and 9.27 min (range, 8.10-12.15 min), respectively (P = 0.001). Highly significant correlations were found between serum prostate-specific antigen value and both PSMA-TV (r = 0.72, P < 0.001) and PSMA-TL (r = 0.66, P = 0.002). Conclusion: Semiautomatic analyses of whole-body tumor burden in 68Ga-PSMA11 PET/CT is feasible. qPSMA is a robust software package that can help physicians quantify tumor load in heavily metastasized prostate cancer patients.

Vertebral labelling and segmentation are two fundamental tasks in an automated spine processing pipeline. Reliable and accurate processing of spine images is expected to benefit clinical decision-support systems for diagnosis, surgery planning, and population-based analysis on spine and bone health. However, designing automated algorithms for spine processing is challenging predominantly due to considerable variations in anatomy and acquisition protocols and due to a severe shortage of publicly available data. Addressing these limitations, the Large Scale Vertebrae Segmentation Challenge (VerSe) was organised in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2019 and 2020, with a call for algorithms towards labelling and segmentation of vertebrae. Two datasets containing a total of 374 multi-detector CT scans from 355 patients were prepared and 4505 vertebrae have individually been annotated at voxel-level by a human-machine hybrid algorithm (https://osf.io/nqjyw/, https://osf.io/t98fz/). A total of 25 algorithms were benchmarked on these datasets. In this work, we present the the results of this evaluation and further investigate the performance-variation at vertebra-level, scan-level, and at different fields-of-view. We also evaluate the generalisability of the approaches to an implicit domain shift in data by evaluating the top performing algorithms of one challenge iteration on data from the other iteration. The principal takeaway from VerSe: the performance of an algorithm in labelling and segmenting a spine scan hinges on its ability to correctly identify vertebrae in cases of rare anatomical variations. The content and code concerning VerSe can be accessed at: https://github.com/anjany/verse., Comment: Challenge report for the VerSe 2019 and 2020. Published in Medical Image Analysis (DOI: https://doi.org/10.1016/j.media.2021.102166)

We present DeepVesselNet, an architecture tailored to the challenges faced when extracting vessel networks or trees and corresponding features in 3-D angiographic volumes using deep learning. We discuss the problems of low execution speed and high memory requirements associated with full 3-D convolutional networks, high-class imbalance arising from the low percentage of vessel voxels, and unavailability of accurately annotated training data - and offer solutions as the building blocks of DeepVesselNet. First, we formulate 2-D orthogonal cross-hair filters which make use of 3-D context information at a reduced computational burden. Second, we introduce a class balancing cross-entropy loss function with false positive rate correction to handle the high-class imbalance and high false positive rate problems associated with existing loss functions. Finally, we generate synthetic dataset using a computational angiogenesis model capable of generating vascular trees under physiological constraints on local network structure and topology and use these data for transfer learning. DeepVesselNet is optimized for segmenting and analyzing vessels, and we test the performance on a range of angiographic volumes including clinical MRA data of the human brain, as well as X-ray tomographic microscopy scans of the rat brain. Our experiments show that, by replacing 3-D filters with cross-hair filters in our network, we achieve over 23% improvement in speed, lower memory footprint, lower network complexity which prevents overfitting and comparable accuracy (with a Cox-Wilcoxon paired sample significance test p-value of 0.07 when compared to full 3-D filters). Our class balancing metric is crucial for training the network and transfer learning with synthetic data is an efficient, robust, and very generalizable approach leading to a network that excels in a variety of angiography segmentation tasks., 13 pages

Objectives To compare spinal bone measures derived from automatic and manual assessment in routine CT with dual energy X-ray absorptiometry (DXA) in their association with prevalent osteoporotic vertebral fractures using our fully automated framework (https://anduin.bonescreen.de) to assess various bone measures in clinical CT. Methods We included 192 patients (141 women, 51 men; age 70.2 ± 9.7 years) who had lumbar DXA and CT available (within 1 year). Automatic assessment of spinal bone measures in CT included segmentation of vertebrae using a convolutional neural network (CNN), reduction to the vertebral body, and extraction of bone mineral content (BMC), trabecular and integral volumetric bone mineral density (vBMD), and CT-based areal BMD (aBMD) using asynchronous calibration. Moreover, trabecular bone was manually sampled (manual vBMD). Results A total of 148 patients (77%) had vertebral fractures and significantly lower values in all bone measures compared to patients without fractures (p ≤ 0.001). Except for BMC, all CT-based measures performed significantly better as predictors for vertebral fractures compared to DXA (e.g., AUC = 0.885 for trabecular vBMD and AUC = 0.86 for integral vBMD vs. AUC = 0.668 for DXA aBMD, respectively; both p < 0.001). Age- and sex-adjusted associations with fracture status were strongest for manual vBMD (OR = 7.3, [95%] CI 3.8–14.3) followed by automatically assessed trabecular vBMD (OR = 6.9, CI 3.5–13.4) and integral vBMD (OR = 4.3, CI 2.5–7.6). Diagnostic cutoffs of integral vBMD for osteoporosis (< 160 mg/cm3) or low bone mass (160 ≤ BMD < 190 mg/cm3) had sensitivity (84%/41%) and specificity (78%/95%) similar to trabecular vBMD. Conclusions Fully automatic osteoporosis screening in routine CT of the spine is feasible. CT-based measures can better identify individuals with reduced bone mass who suffered from vertebral fractures than DXA. Key Points • Opportunistic osteoporosis screening of spinal bone measures derived from clinical routine CT is feasible in a fully automatic fashion using a deep learning-driven framework (https://anduin.bonescreen.de). • Manually sampled volumetric BMD (vBMD) and automatically assessed trabecular and integral vBMD were the best predictors for prevalent vertebral fractures. • Except for bone mineral content, all CT-based bone measures performed significantly better than DXA-based measures. • We introduce diagnostic thresholds of integral vBMD for osteoporosis (< 160 mg/cm3) and low bone mass (160 ≤ BMD < 190 mg/cm3) with almost equal sensitivity and specificity compared to conventional thresholds of quantitative CT as proposed by the American College of Radiology (osteoporosis < 80 mg/cm3).

Tissue clearing methods enable the imaging of biological specimens without sectioning. However, reliable and scalable analysis of large imaging datasets in three dimensions remains a challenge. Here we developed a deep learning-based framework to quantify and analyze brain vasculature, named Vessel Segmentation & Analysis Pipeline (VesSAP). Our pipeline uses a convolutional neural network (CNN) with a transfer learning approach for segmentation and achieves human-level accuracy. By using VesSAP, we analyzed the vascular features of whole C57BL/6J, CD1 and BALB/c mouse brains at the micrometer scale after registering them to the Allen mouse brain atlas. We report evidence of secondary intracranial collateral vascularization in CD1 mice and find reduced vascularization of the brainstem in comparison to the cerebrum. Thus, VesSAP enables unbiased and scalable quantifications of the angioarchitecture of cleared mouse brains and yields biological insights into the vascular function of the brain.

Raster-scan optoacoustic mesoscopy (RSOM) is a powerful, non-invasive optical imaging technique for functional, anatomical, and molecular skin and tissue analysis. However, both the manual and the automated analysis of such images are challenging, because the RSOM images have very low contrast, poor signal to noise ratio, and systematic overlaps between the absorption spectra of melanin and hemoglobin. Nonetheless, the segmentation of the epidermis layer is a crucial step for many downstream medical and diagnostic tasks, such as vessel segmentation or monitoring of cancer progression. We propose a novel, shape-specific loss function that overcomes discontinuous segmentations and achieves smooth segmentation surfaces while preserving the same volumetric Dice and IoU. Further, we validate our epidermis segmentation through the sensitivity of vessel segmentation. We found a 20\(\%\) improvement in Dice for vessel segmentation tasks when the epidermis mask is provided as additional information to the vessel segmentation network.

Despite great advances in brain tumor segmentation and clear clinical need, translation of state-of-the-art computational methods into clinical routine and scientific practice remains a major challenge. Several factors impede successful implementations, including data standardization and preprocessing. However, these steps are pivotal for the deployment of state-of-the-art image segmentation algorithms. To overcome these issues, we present BraTS Toolkit. BraTS Toolkit is a holistic approach to brain tumor segmentation and consists of three components: First, the BraTS Preprocessor facilitates data standardization and preprocessing for researchers and clinicians alike. It covers the entire image analysis workflow prior to tumor segmentation, from image conversion and registration to brain extraction. Second, BraTS Segmentor enables orchestration of BraTS brain tumor segmentation algorithms for generation of fully-automated segmentations. Finally, Brats Fusionator can combine the resulting candidate segmentations into consensus segmentations using fusion methods such as majority voting and iterative SIMPLE fusion. The capabilities of our tools are illustrated with a practical example to enable easy translation to clinical and scientific practice.

SUMMARYTissue clearing methods enable imaging of intact biological specimens without sectioning. However, reliable and scalable analysis of such large imaging data in 3D remains a challenge. Towards this goal, we developed a deep learning-based framework to quantify and analyze the brain vasculature, named Vessel Segmentation & Analysis Pipeline (VesSAP). Our pipeline uses a fully convolutional network with a transfer learning approach for segmentation. We systematically analyzed vascular features of the whole brains including their length, bifurcation points and radius at the micrometer scale by registering them to the Allen mouse brain atlas. We reported the first evidence of secondary intracranial collateral vascularization in CD1-Elite mice and found reduced vascularization in the brainstem as compared to the cerebrum. VesSAP thus enables unbiased and scalable quantifications for the angioarchitecture of the cleared intact mouse brain and yields new biological insights related to the vascular brain function.GRAPHICAL ABSTRACTSupporting material of VesSAP is available athttp://DISCOtechnologies.org/VesSAP

Accurate and automatic organ segmentation is critical for computer-aided analysis towards clinical decision support and treatment planning. State-of-the-art approaches have achieved remarkable segmentation accuracy on large organs, such as the liver and kidneys. However, most of these methods do not perform well on small organs, such as the pancreas, gallbladder, and adrenal glands, especially when lacking sufficient training data. This paper presents an automatic approach for small organ segmentation with limited training data using two cascaded steps—localization and segmentation. The localization stage involves the extraction of the region of interest after the registration of images to a common template and during the segmentation stage, a voxel-wise label map of the extracted region of interest is obtained and then transformed back to the original space. In the localization step, we propose to utilize a graph-based groupwise image registration method to build the template for registration so as to minimize the potential bias and avoid getting a fuzzy template. More importantly, a novel knowledge-aided convolutional neural network is proposed to improve segmentation accuracy in the second stage. This proposed network is flexible and can combine the effort of both deep learning and traditional methods, consequently achieving better segmentation relative to either of individual methods. The ISBI 2015 VISCERAL challenge dataset is used to evaluate the presented approach. Experimental results demonstrate that the proposed method outperforms cutting-edge deep learning approaches, traditional forest-based approaches, and multi-atlas approaches in the segmentation of small organs.

The emerging PSMA-targeted radionuclide therapy provides an effective method for the treatment of advanced metastatic prostate cancer. To optimize the therapeutic effect and maximize the theranostic benefit, there is a need to identify and quantify target lesions prior to treatment. However, this is extremely challenging considering that a high number of lesions of heterogeneous size and uptake may distribute in a variety of anatomical context with different backgrounds. This study proposes an end-to-end deep neural network to characterize the prostate cancer lesions on PSMA imaging automatically. A 68Ga-PSMA-11 PET/CT image dataset including 71 patients with metastatic prostate cancer was collected from three medical centres for training and evaluating the proposed network. For proof-of-concept, we focus on the detection of bone and lymph node lesions in the pelvic area suggestive for metastases of prostate cancer. The preliminary test on pelvic area confirms the potential of deep learning methods. Increasing the amount of training data may further enhance the performance of the proposed deep learning method.

Our aim was to introduce and validate qPSMA, a semiautomatic software package for whole-body tumor burden assessment in prostate cancer patients using (68)Ga-prostate-specific membrane antigen (PSMA) 11 PET/CT. Methods: qPSMA reads hybrid PET/CT images in DICOM format. Its pipeline was written using Python and C++ languages. A bone mask based on CT and a normal-uptake mask including organs with physiologic (68)Ga-PSMA11 uptake are automatically computed. An SUV threshold of 3 and a liver-based threshold are used to segment bone and soft-tissue lesions, respectively. Manual corrections can be applied using different tools. Multiple output parameters are computed, that is, PSMA ligand–positive tumor volume (PSMA-TV), PSMA ligand–positive total lesion (PSMA-TL), PSMA SUV(mean), and PSMA SUV(max). Twenty (68)Ga-PSMA11 PET/CT data sets were used to validate and evaluate the performance characteristics of qPSMA. Four analyses were performed: validation of the semiautomatic algorithm for liver background activity determination, assessment of intra- and interobserver variability, validation of data from qPSMA by comparison with Syngo.via, and assessment of computational time and comparison of PSMA PET–derived parameters with serum prostate-specific antigen. Results: Automatic liver background calculation resulted in a mean relative difference of 0.74% (intraclass correlation coefficient [ICC], 0.996; 95%CI, 0.989;0.998) compared with METAVOL. Intra- and interobserver variability analyses showed high agreement (all ICCs > 0.990). Quantitative output parameters were compared for 68 lesions. Paired t testing showed no significant differences between the values obtained with the 2 software packages. The ICC estimates obtained for PSMA-TV, PSMA-TL, SUV(mean), and SUV(max) were 1.000 (95%CI, 1.000;1.000), 1.000 (95%CI, 1.000;1.000), 0.995 (95%CI, 0.992;0.997), and 0.999 (95%CI, 0.999;1.000), respectively. The first and second reads for intraobserver variability resulted in mean computational times of 13.63 min (range, 8.22–25.45 min) and 9.27 min (range, 8.10–12.15 min), respectively (P = 0.001). Highly significant correlations were found between serum prostate-specific antigen value and both PSMA-TV (r = 0.72, P < 0.001) and PSMA-TL (r = 0.66, P = 0.002). Conclusion: Semiautomatic analyses of whole-body tumor burden in (68)Ga-PSMA11 PET/CT is feasible. qPSMA is a robust software package that can help physicians quantify tumor load in heavily metastasized prostate cancer patients.

The automatic and accurate segmentation of different organs is a critical step for computer-aided diagnosis, treatment planning and clinical decision support. However, for small organs such as the gallbladder, pancreas, and thyroid, accurate segmentation remains challenging due to their limited fraction in the image, high anatomical variability, and inhomogeneity. This paper presents a new fully automated multi-atlas segmentation approach to segment small organs using template construction, robust principal component analysis, and K-nearest neighbor classifier. Qualitative and quantitative evaluation has been evaluated on the VISCERAL challenge dataset. Experimental results show that the proposed system outperforms other multi-atlas based methods and forest-based methods in the segmentation of small organs.

Robust localisation and identification of vertebrae is essential for automated spine analysis. The contribution of this work to the task is two-fold: (1) Inspired by the human expert, we hypothesise that a sagittal and coronal reformation of the spine contain sufficient information for labelling the vertebrae. Thereby, we propose a butterfly-shaped network architecture (termed Btrfly Net) that efficiently combines the information across reformations. (2) Underpinning the Btrfly net, we present an energy-based adversarial training regime that encodes local spine structure as an anatomical prior into the network, thereby enabling it to achieve state-of-art performance in all standard metrics on a benchmark dataset of 302 scans without any post-processing during inference., Published as conference paper in Medical Image Computing and Computer Assisted Intervention - MICCAI 2018

Arterial spin labeling (ASL) allows to quantify the cerebral blood flow (CBF) by magnetic labeling of the arterial blood water. ASL is increasingly used in clinical studies due to its noninvasiveness, repeatability and benefits in quantification. However, ASL suffers from an inherently low-signal-to-noise ratio (SNR) requiring repeated measurements of control/spin-labeled (C/L) pairs to achieve a reasonable image quality, which in return increases motion sensitivity. This leads to clinically prolonged scanning times increasing the risk of motion artifacts. Thus, there is an immense need of advanced imaging and processing techniques in ASL. In this paper, we propose a novel deep learning based approach to improve the perfusion-weighted image quality obtained from a subset of all available pairwise C/L subtractions. Specifically, we train a deep fully convolutional network (FCN) to learn a mapping from noisy perfusion-weighted image and its subtraction (residual) from the clean image. Additionally, we incorporate the CBF estimation model in the loss function during training, which enables the network to produce high quality images while simultaneously enforcing the CBF estimates to be as close as reference CBF values. Extensive experiments on synthetic and clinical ASL datasets demonstrate the effectiveness of our method in terms of improved ASL image quality, accurate CBF parameter estimation and considerably small computation time during testing.

Dynamic contrast-enhanced (DCE) MRI is an evolving imaging technique that provides a quantitative measure of pharmacokinetic (PK) parameters in body tissues, in which series of \(T_1\)-weighted images are collected following the administration of a paramagnetic contrast agent. Unfortunately, in many applications, conventional clinical DCE-MRI suffers from low spatiotemporal resolution and insufficient volume coverage. In this paper, we propose a novel deep learning based approach to directly estimate the PK parameters from undersampled DCE-MRI data. Specifically, we design a custom loss function where we incorporate a forward physical model that relates the PK parameters to corrupted image-time series obtained due to subsampling in k-space. This allows the network to directly exploit the knowledge of true contrast agent kinetics in the training phase, and hence provide more accurate restoration of PK parameters. Experiments on clinical brain DCE datasets demonstrate the efficacy of our approach in terms of fidelity of PK parameter reconstruction and significantly faster parameter inference compared to a model-based iterative reconstruction method.

The identification of bone lesions is crucial in the diagnostic assessment of multiple myeloma (MM). 68Ga-Pentixafor PET/CT can capture the abnormal molecular expression of CXCR-4 in addition to anatomical changes. However, whole-body detection of dozens of lesions on hybrid imaging is tedious and error prone. It is even more difficult to identify lesions with a large heterogeneity. This study employed deep learning methods to automatically combine characteristics of PET and CT for whole-body MM bone lesion detection in a 3D manner. Two convolutional neural networks (CNNs), V-Net and W-Net, were adopted to segment and detect the lesions. The feasibility of deep learning for lesion detection on 68Ga-Pentixafor PET/CT was first verified on digital phantoms generated using realistic PET simulation methods. Then the proposed methods were evaluated on real 68Ga-Pentixafor PET/CT scans of MM patients. The preliminary results showed that deep learning method can leverage multimodal information for spatial feature representation, and W-Net obtained the best result for segmentation and lesion detection. It also outperformed traditional machine learning methods such as random forest classifier (RF), k-Nearest Neighbors (k-NN), and support vector machine (SVM). The proof-of-concept study encourages further development of deep learning approach for MM lesion detection in population study.

Feature extraction is a very crucial task in image and pixel (voxel) classification and regression in biomedical image modeling. In this work we present a machine learning based feature extraction scheme based on inception models for pixel classification tasks. We extract features under multi-scale and multi-layer schemes through convolutional operators. Layers of Fully Convolutional Network are later stacked on this feature extraction layers and trained end-to-end for the purpose of classification. We test our model on the DRIVE and STARE public data sets for the purpose of segmentation and centerline detection and it out performs most existing hand crafted or deterministic feature schemes found in literature. We achieve an average maximum Dice of 0.85 on the DRIVE data set which out performs the scores from the second human annotator of this data set. We also achieve an average maximum Dice of 0.85 and kappa of 0.84 on the STARE data set. Though these datasets are mainly 2-D we also propose ways of extending this feature extraction scheme to handle 3-D datasets.

In this paper, we present a multi-modal medical image classification framework classifying brain tumor glioblastomas in genetic classes based on DNA methylation status. The framework makes use of computationally efficient 3D implementations of short local image descriptors, such as LBP, BRIEF and HOG, which are processed by a Bag-of-Patterns model to represent image regions, as well as deep-learned features acquired by denoising auto-encoders and hand-crafted shape features calculated on segmentation masks. The framework is validated against a cohort of 116 brain tumor patients from the TCIA database and is shown to obtain high accuracies even though the same image-based classification task is hardly possible for medical experts.

The assessment of bone lesion is crucial for the diagnostic and therapeutic planning of multiple myeloma (MM). \(^{68}\)Ga-Pentixafor PET/CT can capture the abnormal molecular expression of CXCR-4 in addition to anatomical changes. However, the whole-body detection of dozens of lesions on hybrid imaging is tedious and error-prone. In this paper, we adopt a cascaded convolutional neural networks (CNN) to form a W-shaped architecture (W-Net). This deep learning method leverages multimodal information for lesion detection. The first part of W-Net extracts skeleton from CT scan and the second part detect and segment lesions. The network was tested on 12 \(^{68}\)Ga-Pentixafor PET/CT scans of MM patients using 3-folder cross validation. The preliminary results showed that W-Net can automatically learn features from multimodal imaging for MM bone lesion detection. The proof-of-concept study encouraged further development of deep learning approach for MM lesion detection with increased number of subjects.