Search

This talk provides a quick overview of ilastik from both user and developer perspective. We also discuss the current developments and plans of immediate future, including remote server, alternative viewer and more stream-lined distributed computation.

Lateral root formation determines to a large extent the ability of plants to forage their environment and thus their growth. In Arabidopsis thaliana and other angiosperms, lateral root initiation requires radial cell expansion and several rounds of anticlinal cell divisions that give rise to a central core of small pericycle cells, which express different markers than the larger surrounding cells. These small central cells then switch their plane of divisions to periclinal, and give rise to seemingly morphologically similar daughter cells that have different identities and establish the different cell types of the new root. Although the execution of these two types of divisions is tightly regulated and essential for the correct development of the lateral root, we know little about their geometrical features. Here we analyse a four-dimensional reconstruction of the first stages of lateral root formation and analyze the geometric features of the anticlinal and periclinal divisions. We identify that the periclinal divisions of the small central cells are morphologically dissimilar and asymmetric. We show that mother cell volume is different when looking at anticlinal versus periclinal divisions and the repeated anticlinal divisions do not lead to reduction in cell volume although cells are shorter. Finally, we show that cells undergoing a periclinal division are characterized by a strong cell expansion. Our results indicate that cells integrate growth and division to precisely partition their volume upon division during the first two stages of lateral root formation.

The intrinsic genetic programme of a cell is not sufficient to explain all of the cell’s activities. External mechanical stimuli are increasingly recognized as determinants of cell behaviour. In the epithelial folding event that constitutes the beginning of gastrulation inDrosophila, the genetic programme of the future mesoderm leads to the establishment of a contractile actomyosin network that triggers apical constriction of cells, and thereby, tissue folding. However, some cells do not constrict but instead stretch, even though they share the same genetic programme as their constricting neighbours. We show here that tissue-wide interactions force these cells to expand even when an otherwise sufficient amount of apical, active actomyosin is present. Models based on contractile forces and linear stress-strain responses do not reproduce experimental observations, but simulations in which cells behave as ductile materials with non-linear mechanical properties do. Our models show that this behaviour is a general emergent property of actomyosin networks [in a supracellular context, in accordance with our experimental observations of actin reorganisation within stretching cells.

Emergence of the novel pathogenic coronavirus SARS‐CoV‐2 and its rapid pandemic spread presents challenges that demand immediate attention. Here, we describe the development of a semi‐quantitative high‐content microscopy‐based assay for detection of three major classes (IgG, IgA, and IgM) of SARS‐CoV‐2 specific antibodies in human samples. The possibility to detect antibodies against the entire viral proteome together with a robust semi‐automated image analysis workflow resulted in specific, sensitive and unbiased assay that complements the portfolio of SARS‐CoV‐2 serological assays. Sensitive, specific and quantitative serological assays are urgently needed for a better understanding of humoral immune response against the virus as a basis for developing public health strategies to control viral spread. The procedure described here has been used for clinical studies and provides a general framework for the application of quantitative high‐throughput microscopy to rapidly develop serological assays for emerging virus infections.

A fundamental question in biology is how morphogenesis integrates the multitude of processes that act at different scales, ranging from the molecular control of gene expression to cellular coordination in a tissue. Using machine-learning-based digital image analysis, we generated a three-dimensional atlas of ovule development in Arabidopsis thaliana, enabling the quantitative spatio-temporal analysis of cellular and gene expression patterns with cell and tissue resolution. We discovered novel morphological manifestations of ovule polarity, a new mode of cell layer formation, and previously unrecognized subepidermal cell populations that initiate ovule curvature. The data suggest an irregular cellular build-up of WUSCHEL expression in the primordium and new functions for INNER NO OUTER in restricting nucellar cell proliferation and the organization of the interior chalaza. Our work demonstrates the analytical power of a three-dimensional digital representation when studying the morphogenesis of an organ of complex architecture that eventually consists of 1900 cells.

Visualizing dynamic processes over large, three-dimensional fields of view at high speed is essential for many applications in the life sciences. Light-field microscopy (LFM) has emerged as a tool for fast volumetric image acquisition, but its effective throughput and widespread use in biology has been hampered by a computationally demanding and artifact-prone image reconstruction process. Here, we present a framework for artificial intelligence-enhanced microscopy, integrating a hybrid light-field light-sheet microscope and deep learning-based volume reconstruction. In our approach, concomitantly acquired, high-resolution two-dimensional light-sheet images continuously serve as training data and validation for the convolutional neural network reconstructing the raw LFM data during extended volumetric time-lapse imaging experiments. Our network delivers high-quality three-dimensional reconstructions at video-rate throughput, which can be further refined based on the high-resolution light-sheet images. We demonstrate the capabilities of our approach by imaging medaka heart dynamics and zebrafish neural activity with volumetric imaging rates up to 100 Hz.

Unbiased quantitative analysis of macroscopic biological samples demands fast imaging systems capable of maintaining high resolution across large volumes. Here we introduce RAPID (rapid autofocusing via pupil-split image phase detection), a real-time autofocus method applicable in every widefield-based microscope. RAPID-enabled light-sheet microscopy reliably reconstructs intact, cleared mouse brains with subcellular resolution, and allowed us to characterize the three-dimensional (3D) spatial clustering of somatostatin-positive neurons in the whole encephalon, including densely labeled areas. Furthermore, it enabled 3D morphological analysis of microglia across the entire brain. Beyond light-sheet microscopy, we demonstrate that RAPID maintains high image quality in various settings, from in vivo fluorescence imaging to 3D tracking of fast-moving organisms. RAPID thus provides a flexible autofocus solution that is suitable for traditional automated microscopy tasks as well as for quantitative analysis of large biological specimens.

Quantitative analysis of plant and animal morphogenesis requires accurate segmentation of individual cells in volumetric images of growing organs. In the last years, deep learning has provided robust automated algorithms that approach human performance, with applications to bio-image analysis now starting to emerge. Here, we present PlantSeg, a pipeline for volumetric segmentation of plant tissues into cells. PlantSeg employs a convolutional neural network to predict cell boundaries and graph partitioning to segment cells based on the neural network predictions. PlantSeg was trained on fixed and live plant organs imaged with confocal and light sheet microscopes. PlantSeg delivers accurate results and generalizes well across different tissues, scales, acquisition settings even on non plant samples. We present results of PlantSeg applications in diverse developmental contexts. PlantSeg is free and open-source, with both a command line and a user-friendly graphical interface.

Emergence of the novel pathogenic coronavirus Sars-CoV-2 and its rapid pandemic spread presents numerous questions and challenges that demand immediate attention. Among these is the urgent need for a better understanding of humoral immune response against the virus and assessment of seroprevalence levels in the population, both of which form the basis for developing public health strategies to control viral spread. For this sensitive, specific and quantitative serological assays are required. Here, we describe the development of a semi-quantitative high-content microscopy-based assay for detection of three major classes (IgG, IgA and IgM) of Sars-CoV-2 specific antibodies in human samples. The possibility to detect antibodies against the entire viral proteome together with a robust semi-automated image analysis workflow resulted in improvement of sensitivity and specificity compared to an approved ELISA-based diagnostic test. Combining both resulted in maximum specificity in a negative control cohort, while maintaining high sensitivity. The procedure described here is compatible with high-throughput microscopy approaches and may be applied for serological analysis of other virus infections.

SummaryAnimal bodies are composed of hundreds of cell types that differ in location, morphology, cytoarchitecture, and physiology. This is reflected by cell type-specific transcription factors and downstream effector genes implementing functional specialisation. Here, we establish and explore the link between cell type-specific gene expression and subcellular morphology for the entire body of the marine annelidPlatynereis dumerilii. For this, we registered a whole-body cellular expression atlas to a high-resolution electron microscopy dataset, automatically segmented all cell somata and nuclei, and clustered the cells according to gene expression or morphological parameters. We show that collective gene expression most efficiently identifies spatially coherent groups of cells that match anatomical boundaries, which indicates that combinations of regionally expressed transcription factors specify tissue identity. We provide an integrated browser as a Fiji plugin to readily explore, analyse and visualise multimodal datasets with remote on-demand access to all available datasets.

Quantitative analysis of plant and animal morphogenesis requires accurate segmentation of individual cells in volumetric images of growing organs. In the last years, deep learning has provided robust automated algorithms that approach human performance, with applications to bio-image analysis now starting to emerge. Here, we present PlantSeg, a pipeline for volumetric segmentation of plant tissues into cells. PlantSeg employs a convolutional neural network to predict cell boundaries and graph partitioning to segment cells based on the neural network predictions. PlantSeg was trained on fixed and live plant organs imaged with confocal and light sheet microscopes. PlantSeg delivers accurate results and generalizes well across different tissues, scales, and acquisition settings. We present results of PlantSeg applications in diverse developmental contexts. PlantSeg is free and open-source, with both a command line and a user-friendly graphical interface.

Semantic instance segmentation is the task of simultaneously partitioning an image into distinct segments while associating each pixel with a class label. In commonly used pipelines, segmentation and label assignment are solved separately since joint optimization is computationally expensive. We propose a greedy algorithm for joint graph partitioning and labeling derived from the efficient Mutex Watershed partitioning algorithm. It optimizes an objective function closely related to the Asymmetric Multiway Cut objective and empirically shows efficient scaling behavior. Due to the algorithm’s efficiency it can operate directly on pixels without prior over-segmentation of the image into superpixels. We evaluate the performance on the Cityscapes dataset (2D urban scenes) and on a 3D microscopy volume. In urban scenes, the proposed algorithm combined with current deep neural networks outperforms the strong baseline of ‘Panoptic Feature Pyramid Networks’ by Kirillov et al. (2019). In the 3D electron microscopy images, we show explicitly that our joint formulation outperforms a separate optimization of the partitioning and labeling problems.

In the version of this Article initially published, Jacob H. Macke and Jonas Ries were not listed as corresponding authors. Their contact information and designation as corresponding authors are now included. The error has been corrected in the online version of the Article.

The throughput of electron microscopes has increased significantly in recent years, enabling detailed analysis of cell morphology and ultrastructure in fairly large tissue volumes. Analysis of neural circuits at single-synapse resolution remains the flagship target of this technique, but applications to cell and developmental biology are also starting to emerge at scale. On the light microscopy side, continuous development of light-sheet microscopes has led to a rapid increase in imaged volume dimensions, making Terabyte-scale acquisitions routine in the field. The amount of data acquired in such studies makes manual instance segmentation, a fundamental step in many analysis pipelines, impossible. While automatic segmentation approaches have improved significantly thanks to the adoption of convolutional neural networks, their accuracy still lags behind human annotations and requires additional manual proof-reading. A major hindrance to further improvements is the limited field of view of the segmentation networks preventing them from learning to exploit the expected cell morphology or other prior biological knowledge which humans use to inform their segmentation decisions. In this contribution, we show how such domain-specific information can be leveraged by expressing it as long-range interactions in a graph partitioning problem known as the lifted multicut problem. Using this formulation, we demonstrate significant improvement in segmentation accuracy for four challenging boundary-based segmentation problems from neuroscience and developmental biology.

The evolutionary origin of metazoan cell types such as neurons, muscles, digestive, and immune cells, remains unsolved. Using whole-body single-cell RNA sequencing in a sponge, an animal without nervous system and musculature, we identify 18 distinct cell types comprising four major families. This includes nitric-oxide sensitive contractile cells, digestive cells active in macropinocytosis, and a family of amoeboid-neuroid cells involved in innate immunity. We uncover ‘presynaptic’ genes in an amoeboid-neuroid cell type, and ‘postsynaptic’ genes in digestive choanocytes, suggesting asymmetric and targeted communication. Corroborating this, long neurite-like extensions from neuroid cells directly contact and enwrap choanocyte microvillar collars. Our data indicate a link between neuroid and immune functions in sponges, and suggest that a primordial neuro-immune system cleared intruders and controlled ciliary beating for feeding.

Segmentation is one of the most ubiquitous problems in biological image analysis. Here we present a machine learning-based solution to it as implemented in the open source ilastik toolkit. We give a broad description of the underlying theory and demonstrate two workflows: Pixel Classification and Autocontext. We illustrate their use on a challenging problem in electron microscopy image segmentation. After following this walk-through, we expect the readers to be able to apply the necessary steps to their own data and segment their images by either workflow.

Recent advances in high-throughput electron microscopy imaging enable detailed study of centrosome aberrations in cancer cells. While the image acquisition in such pipelines is automated, manual detection of centrioles is still necessary to select cells for re-imaging at higher magnification. In this contribution we propose an algorithm which performs this step automatically and with high accuracy. From the image labels produced by human experts and a 3D model of a centriole we construct an additional training set with patch-level labels. A two-level DenseNet is trained on the hybrid training data with synthetic patches and real images, achieving much better results on real patient data than training only at the image-level. The code can be found at https://github.com/kreshuklab/centriole_detection.

In the last years, automated segmentation has become a necessary tool for volume electron microscopy (EM) imaging. So far, the best performing techniques have been largely based on fully supervised encoder-decoder CNNs, requiring a substantial amount of annotated images. Domain Adaptation (DA) aims to alleviate the annotation burden by 'adapting' the networks trained on existing groundtruth data (source domain) to work on a different (target) domain with as little additional annotation as possible. Most DA research is focused on the classification task, whereas volume EM segmentation remains rather unexplored. In this work, we extend recently proposed classification DA techniques to an encoder-decoder layout and propose a novel method that adds a reconstruction decoder to the classical encoder-decoder segmentation in order to align source and target encoder features. The method has been validated on the task of segmenting mitochondria in EM volumes. We have performed DA from brain EM images to HeLa cells and from isotropic FIB/SEM volumes to anisotropic TEM volumes. In all cases, the proposed method has outperformed the extended classification DA techniques and the finetuning baseline. An implementation of our work can be found on https://github.com/JorisRoels/domain-adaptive-segmentation., Comment: ISBI 2019 (accepted)

We propose a theoretical framework that generalizes simple and fast algorithms for hierarchical agglomerative clustering to weighted graphs with both attractive and repulsive interactions between the nodes. This framework defines GASP, a Generalized Algorithm for Signed graph Partitioning, and allows us to explore many combinations of different linkage criteria and cannot-link constraints. We prove the equivalence of existing clustering methods to some of those combinations and introduce new algorithms for combinations that have not been studied before. We study both theoretical and empirical properties of these combinations and prove that some of these define an ultrametric on the graph. We conduct a systematic comparison of various instantiations of GASP on a large variety of both synthetic and existing signed clustering problems, in terms of accuracy but also efficiency and robustness to noise. Lastly, we show that some of the algorithms included in our framework, when combined with the predictions from a CNN model, result in a simple bottom-up instance segmentation pipeline. Going all the way from pixels to final segments with a simple procedure, we achieve state-of-the-art accuracy on the CREMI 2016 EM segmentation benchmark without requiring domain-specific superpixels., Comment: Published in CVPR 2022

Segmentation is one of the most ubiquitous problems in biological image analysis. Here we present a machine learning-based solution to it as implemented in the open source ilastik toolkit. We give a broad description of the underlying theory and demonstrate two workflows: Pixel Classification and Autocontext. We illustrate their use on a challenging problem in electron microscopy image segmentation. After following this walk-through, we expect the readers to be able to apply the necessary steps to their own data and segment their images by either workflow.

Image partitioning, or segmentation without semantics, is the task of decomposing an image into distinct segments, or equivalently to detect closed contours. Most prior work either requires seeds, one per segment; or a threshold; or formulates the task as multicut / correlation clustering, an NP-hard problem. Here, we propose an efficient algorithm for graph partitioning, the “Mutex Watershed”. Unlike seeded watershed, the algorithm can accommodate not only attractive but also repulsive cues, allowing it to find a previously unspecified number of segments without the need for explicit seeds or a tunable threshold. We also prove that this simple algorithm solves to global optimality an objective function that is intimately related to the multicut / correlation clustering integer linear programming formulation. The algorithm is deterministic, very simple to implement, and has empirically linearithmic complexity. When presented with short-range attractive and long-range repulsive cues from a deep neural network, the Mutex Watershed gives the best results currently known for the competitive ISBI 2012 EM segmentation benchmark.

Image partitioning, or segmentation without semantics, is the task of decomposing an image into distinct segments; or equivalently, the task of detecting closed contours in an image. Most prior work either requires seeds, one per segment; or a threshold; or formulates the task as an NP-hard signed graph partitioning problem. Here, we propose an algorithm with empirically linearithmic complexity. Unlike seeded watershed, the algorithm can accommodate not only attractive but also repulsive cues, allowing it to find a previously unspecified number of segments without the need for explicit seeds or a tunable threshold. The algorithm itself, which we dub “Mutex Watershed”, is closely related to a minimal spanning tree computation. It is deterministic and easy to implement. When presented with short-range attractive and long-range repulsive cues from a deep neural network, the Mutex Watershed gives results that currently define the state-of-the-art in the competitive ISBI 2012 EM segmentation benchmark. These results are also better than those obtained from other recently proposed clustering strategies operating on the very same network outputs.