Search

The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

We present a novel deep learning approach to categorical segmentation of lung CTs of COVID-19 patients. Specifically, we partition the scans into healthy lung tissues, non-lung regions, and two different, yet visually similar, pathological lung tissues, namely, ground-glass opacity and consolidation. This is accomplished via a unique, end-to-end hierarchical network architecture and ensemble learning, which contribute to the segmentation and provide a measure for segmentation uncertainty. The proposed framework achieves competitive results and outstanding generalization capabilities for three COVID-19 datasets. Our method is ranked second in a public Kaggle competition for COVID-19 CT images segmentation. Moreover, segmentation uncertainty regions are shown to correspond to the disagreements between the manual annotations of two different radiologists. Finally, preliminary promising correspondence results are shown for our private dataset when comparing the patients' COVID-19 severity scores (based on clinical measures), and the segmented lung pathologies. Code and data are available at our repository: https://github.com/talbenha/covid-seg, Comment: IEEE International Conference on Image Processing (ICIP) 2022

Repetitive mild traumatic brain injury in American football players has garnered increasing public attention following reports of chronic traumatic encephalopathy, a progressive tauopathy. While the mechanisms underlying repetitive mild traumatic brain injury-induced neurodegeneration are unknown and antemortem diagnostic tests are not available, neuropathology studies suggest a pathogenic role for microvascular injury, specifically blood-brain barrier dysfunction. Thus, our main objective was to demonstrate the effectiveness of a modified dynamic contrast-enhanced MRI approach we have developed to detect impairments in brain microvascular function. To this end, we scanned 42 adult male amateur American football players and a control group comprising 27 athletes practicing a non-contact sport and 26 non-athletes. MRI scans were also performed in 51 patients with brain pathologies involving the blood-brain barrier, namely malignant brain tumours, ischaemic stroke and haemorrhagic traumatic contusion. Based on data from prolonged scans, we generated maps that visualized the permeability value for each brain voxel. Our permeability maps revealed an increase in slow blood-to-brain transport in a subset of amateur American football players, but not in sex- and age-matched controls. The increase in permeability was region specific (white matter, midbrain peduncles, red nucleus, temporal cortex) and correlated with changes in white matter, which were confirmed by diffusion tensor imaging. Additionally, increased permeability persisted for months, as seen in players who were scanned both on- and off-season. Examination of patients with brain pathologies revealed that slow tracer accumulation characterizes areas surrounding the core of injury, which frequently shows fast blood-to-brain transport. Next, we verified our method in two rodent models: rats and mice subjected to repeated mild closed-head impact injury, and rats with vascular injury inflicted by photothrombosis. In

Repetitive mild traumatic brain injury in American football players has garnered increasing public attention following reports of chronic traumatic encephalopathy, a progressive tauopathy. While the mechanisms underlying repetitive mild traumatic brain injury-induced neurodegeneration are unknown and antemortem diagnostic tests are not available, neuropathology studies suggest a pathogenic role for microvascular injury, specifically blood-brain barrier dysfunction. Thus, our main objective was to demonstrate the effectiveness of a modified dynamic contrast-enhanced MRI approach we have developed to detect impairments in brain microvascular function. To this end, we scanned 42 adult male amateur American football players and a control group comprising 27 athletes practicing a non-contact sport and 26 non-athletes. MRI scans were also performed in 51 patients with brain pathologies involving the blood-brain barrier, namely malignant brain tumours, ischaemic stroke and haemorrhagic traumatic contusion. Based on data from prolonged scans, we generated maps that visualized the permeability value for each brain voxel. Our permeability maps revealed an increase in slow blood-to-brain transport in a subset of amateur American football players, but not in sex- and age-matched controls. The increase in permeability was region specific (white matter, midbrain peduncles, red nucleus, temporal cortex) and correlated with changes in white matter, which were confirmed by diffusion tensor imaging. Additionally, increased permeability persisted for months, as seen in players who were scanned both on- and off-season. Examination of patients with brain pathologies revealed that slow tracer accumulation characterizes areas surrounding the core of injury, which frequently shows fast blood-to-brain transport. Next, we verified our method in two rodent models: rats and mice subjected to repeated mild closed-head impact injury, and rats with vascular injury inflicted by photothrombosis. In

The pressing need to reduce the capacity of deep neural networks has stimulated the development of network dilution methods and their analysis. While the ability of $L_1$ and $L_0$ regularization to encourage sparsity is often mentioned, $L_2$ regularization is seldom discussed in this context. We present a novel framework for weight pruning by sampling from a probability function that favors the zeroing of smaller weights. In addition, we examine the contribution of $L_1$ and $L_2$ regularization to the dynamics of node pruning while optimizing for weight pruning. We then demonstrate the effectiveness of the proposed stochastic framework when used together with a weight decay regularizer on popular classification models in removing 50% of the nodes in an MLP for MNIST classification, 60% of the filters in VGG-16 for CIFAR10 classification, and on medical image models in removing 60% of the channels in a U-Net for instance segmentation and 50% of the channels in CNN model for COVID-19 detection. For these node-pruned networks, we also present competitive weight pruning results that are only slightly less accurate than the original, dense networks.

The problem of inter-rater variability is often discussed in the context of manual labeling of medical images. The emergence of data-driven approaches such as Deep Neural Networks (DNNs) brought this issue of raters' disagreement to the front-stage. In this paper, we highlight the issue of inter-rater bias as opposed to random inter-observer variability and demonstrate its influence on DNN training, leading to different segmentation results for the same input images. In fact, lower overlap scores are obtained between the outputs of a DNN trained on annotations of one rater and tested on another. Moreover, we demonstrate that inter-rater bias in the training examples is amplified and becomes more consistent, considering the segmentation predictions of the DNNs' test data. We support our findings by showing that a classifier-DNN trained to distinguish between raters based on their manual annotations performs better when the automatic segmentation predictions rather than the actual raters' annotations were tested. For this study, we used two different datasets: the ISBI 2015 Multiple Sclerosis (MS) challenge dataset, including MRI scans each with annotations provided by two raters with different levels of expertise; and Intracerebral Hemorrhage (ICH) CT scans with manual and semi-manual segmentations. The results obtained allow us to underline a worrisome clinical implication of a DNN bias induced by an inter-rater bias during training. Specifically, we present a consistent underestimate of MS-lesion loads when calculated from segmentation predictions of a DNN trained on input provided by the less experienced rater. In the same manner, the differences in ICH volumes calculated based on outputs of identical DNNs, each trained on annotations from a different source are more consistent and larger than the differences in volumes between the manual and semi-manual annotations used for training.

We present DeepTract, a deep-learning framework for estimating white matter fibers orientation and streamline tractography. We adopt a data-driven approach for fiber reconstruction from diffusion weighted images (DWI), which does not assume a specific diffusion model. We use a recurrent neural network for mapping sequences of DWI values into probabilistic fiber orientation distributions. Based on these estimations, our model facilitates both deterministic and probabilistic streamline tractography. We quantitatively evaluate our method using the Tractometer tool, demonstrating competitive performance with state-of-the art classical and machine learning based tractography algorithms. We further present qualitative results of bundle-specific probabilistic tractography obtained using our method. The code is publicly available at: https://github.com/itaybenou/DeepTract.git.

We introduce a generative probabilistic model for segmentation of brain lesions in multi-dimensional images that generalizes the EM segmenter, a common approach for modelling brain images using Gaussian mixtures and a probabilistic tissue atlas that employs expectation-maximization (EM), to estimate the label map for a new image. Our model augments the probabilistic atlas of the healthy tissues with a latent atlas of the lesion. We derive an estimation algorithm with closed-form EM update equations. The method extracts a latent atlas prior distribution and the lesion posterior distributions jointly from the image data. It delineates lesion areas individually in each channel, allowing for differences in lesion appearance across modalities, an important feature of many brain tumor imaging sequences. We also propose discriminative model extensions to map the output of the generative model to arbitrary labels with semantic and biological meaning, such as “tumor core” or “fluid-filled structure”, but without a one-to-one correspondence to the hypo- or hyper-intense lesion areas identified by the generative model. We test the approach in two image sets: the publicly available BRATS set of glioma patient scans, and multimodal brain images of patients with acute and subacute ischemic stroke. We find the generative model that has been designed for tumor lesions to generalize well to stroke images, and the extended discriminative -discriminative model to be one of the top ranking methods in the BRATS evaluation., National Alliance for Medical Image Computing (U.S.) (NIH NIBIB NAMIC U54- EB005149), Neuroimaging Analysis Center (U.S.) (NIH NIBIB NAC P41EB015902), National Institute for Biomedical Imaging and Bioengineering (U.S.) (R01EB013565), Lundbeck Foundation (R141-2013-13117)

We introduce a generative probabilistic model for segmentation of brain lesions in multi-dimensional images that generalizes the EM segmenter, a common approach for modelling brain images using Gaussian mixtures and a probabilistic tissue atlas that employs expectation-maximization (EM), to estimate the label map for a new image. Our model augments the probabilistic atlas of the healthy tissues with a latent atlas of the lesion. We derive an estimation algorithm with closed-form EM update equations. The method extracts a latent atlas prior distribution and the lesion posterior distributions jointly from the image data. It delineates lesion areas individually in each channel, allowing for differences in lesion appearance across modalities, an important feature of many brain tumor imaging sequences. We also propose discriminative model extensions to map the output of the generative model to arbitrary labels with semantic and biological meaning, such as "tumor core" or "fluid-filled structure", but without a one-to-one correspondence to the hypo- or hyper-intense lesion areas identified by the generative model. We test the approach in two image sets: the publicly available BRATS set of glioma patient scans, and multimodal brain images of patients with acute and subacute ischemic stroke. We find the generative model that has been designed for tumor lesions to generalize well to stroke images, and the extended discriminative-discriminative model to be one of the top ranking methods in the BRATS evaluation.

Spatial priors, such as probabilistic atlases, play an important role in MRI segmentation. However, the availability of comprehensive, reliable and suitable manual segmentations for atlas construction is limited. We therefore propose a method for joint segmentation of corresponding regions of interest in a collection of aligned images that does not require labeled training data. Instead, a latent atlas, initialized by at most a single manual segmentation, is inferred from the evolving segmentations of the ensemble. The algorithm is based on probabilistic principles but is solved using partial differential equations (PDEs) and energy minimization criteria. We evaluate the method on two datasets, segmenting subcortical and cortical structures in a multi-subject study and extracting brain tumors in a single-subject multi-modal longitudinal experiment. We compare the segmentation results to manual segmentations, when those exist, and to the results of a state-of-the-art atlas-based segmentation method. The quality of the results supports the latent atlas as a promising alternative when existing atlases are not compatible with the images to be segmented., National Institutes of Health (U.S.) (National Institute for Biomedical Imaging and Bioengineering (U.S.)/National Alliance for Medical Image Computing (U.S.) U54-EB005149), National Institutes of Health (U.S.) (National Center for Research Resources (U.S.)/Neuroimaging Analysis Center (U.S.) P41-RR13218), National Institutes of Health (U.S.) (National Institute of Neurological Disorders and Stroke (U.S.) R01-NS051826), National Institutes of Health (U.S.) (National Center for Research Resources (U.S.)/Biomedical Informatics Research Network U24-RR021382), National Science Foundation (U.S.) (CAREER Award 0642971), German Academy of Sciences Leopoldina (Fellowship LPDS 2009-10), Academy of Finland (Grant 133611)

We present a toolbox for high-throughput screening of image-based Caenorhabditis elegans phenotypes. The image analysis algorithms measure morphological phenotypes in individual worms and are effective for a variety of assays and imaging systems from different laboratories. The toolbox is available via the open-source CellProfiler project and enables objective scoring of whole-animal high-throughput image-based assays using this unique model organism for the study of diverse biological pathways relevant to human disease.

The roundworm Caenorhabditis elegans is an effective model system for biological processes such as immunity, behavior, and metabolism. Robotic sample preparation together with automated microscopy and image analysis has recently enabled high-throughput screening experiments using C. elegans. So far, such experiments have been limited to per-image measurements due to the tendency of the worms to cluster, which prevents extracting features from individual animals. We present a novel approach for the extraction of individual C. elegans from clusters of worms in high-throughput microscopy images. The key ideas are the construction of a low-dimensional shape-descriptor space and the definition of a probability measure on it. Promising segmentation results are shown., National Institutes of Health (U.S.) (Grant R01 AI072508), National Institutes of Health (U.S.) (Grant P01 AI083214), National Institutes of Health (U.S.) (Grant R01 AI085581), National Institutes of Health (U.S.) (Grant U54 EB005149)

The roundworm Caenorhabditis elegans is an effective model system for biological processes such as immunity, behavior, and metabolism. Robotic sample preparation together with automated microscopy and image analysis has recently enabled high-throughput screening experiments using C. elegans. So far, such experiments have been limited to per-image measurements due to the tendency of the worms to cluster, which prevents extracting features from individual animals. We present a novel approach for the extraction of individual C. elegans from clusters of worms in high-throughput microscopy images. The key ideas are the construction of a low-dimensional shape-descriptor space and the definition of a probability measure on it. Promising segmentation results are shown., National Institutes of Health (U.S.) (Grant R01 AI072508), National Institutes of Health (U.S.) (Grant P01 AI083214), National Institutes of Health (U.S.) (Grant R01 AI085581), National Institutes of Health (U.S.) (Grant U54 EB005149)

We introduce a novel variational method for the extraction of objects with either bilateral or rotational symmetry in the presence of perspective distortion. Information on the symmetry axis of the object and the distorting transformation is obtained as a by-product of the segmentation process. The key idea is the use of a flip or a rotation of the image to segment as if it were another view of the object. We call this generated image the symmetrical counterpart image. We show that the symmetrical counterpart image and the source image are related by planar projective homography. This homography is determined by the unknown planar projective transformation that distorts the object symmetry. The proposed segmentation method uses a level-set-based curve evolution technique. The extraction of the object boundaries is based on the symmetry constraint and the image data. The symmetrical counterpart of the evolving level-set function provides a dynamic shape prior. It supports the segmentation by resolving possible ambiguities due to noise, clutter, occlusions, and assimilation with the background. The homography that aligns the symmetrical counterpart to the source level-set is recovered via a registration process carried out concurrently with the segmentation. Promising segmentation results of various images of approximately symmetrical objects are shown., MUSCLE Network of Excellence, Yizhak and Chaya Weinstein Research Institute for Signal Processing, Universiṭat Tel-Aviv. A.M.N. Foundation

Image-guided therapy procedures require the patient to remain still throughout the image acquisition, data analysis and therapy. This imposes a tight time constraint on the over-all process. Automatic extraction of the pathological regions prior to the therapy can be faster than the customary manual segmentation performed by the physician. However, the image data alone is usually not sufficient for reliable and unambiguous computerized segmentation. Thus, the oversight of an experienced physician remains mandatory. We present a novel segmentation framework, that allows user feedback. A few mouse-clicks of the user, discrete in nature, are represented as a continuous energy term that is incorporated into a level-set functional. We demonstrate the proposed method on MR scans of uterine fibroids acquired prior to focused ultrasound ablation treatment. The experiments show that with a minimal user input, automatic segmentation results become practically identical to manual expert segmentation., A.M.N. Foundation for the Advancement of Science, Art and Culture in Israel