Search

The cell is arguably the most fundamental unit of life and is central to understanding biology. Accurate modeling of cells is important for this understanding as well as for determining the root causes of disease. Recent advances in artificial intelligence (AI), combined with the ability to generate large-scale experimental data, present novel opportunities to model cells. Here we propose a vision of leveraging advances in AI to construct virtual cells, high-fidelity simulations of cells and cellular systems under different conditions that are directly learned from biological data across measurements and scales. We discuss desired capabilities of such AI Virtual Cells, including generating universal representations of biological entities across scales, and facilitating interpretable in silico experiments to predict and understand their behavior using Virtual Instruments. We further address the challenges, opportunities and requirements to realize this vision including data needs, evaluation strategies, and community standards and engagement to ensure biological accuracy and broad utility. We envision a future where AI Virtual Cells help identify new drug targets, predict cellular responses to perturbations, as well as scale hypothesis exploration. With open science collaborations across the biomedical ecosystem that includes academia, philanthropy, and the biopharma and AI industries, a comprehensive predictive understanding of cell mechanisms and interactions has come into reach., Competing Interests: Competing interests C.B. and A. R. are employees of Genentech, a member of the Roche Group. A.R. has equity in Roche. A.R. was a co-founder and equity holder of Celsius Therapeutics, and is an equity holder in Immunitas. Until July 31, 2020 A.R. was an S.A.B. member of ThermoFisher Scientific, Syros Pharmaceuticals, Neogene Therapeutics and Asimov. A.R. is a named inventor on multiple filed patents related to single cell and spatial genomics, including for scRNA-seq, spatial transcriptomics, Perturb-Seq, compressed experiments, and PerturbView. E.L. is an advisor for the Chan-Zuckerberg Initiative Foundation. N.J.S. is an employee of EvolutionaryScale, PBC.

Single-molecule localization microscopy (SMLM) has had remarkable success in imaging cellular structures with nanometer resolution, but standard analysis algorithms require sparse emitters, which limits imaging speed and labeling density. Here, we overcome this major limitation using deep learning. We developed DECODE (deep context dependent), a computational tool that can localize single emitters at high density in three dimensions with highest accuracy for a large range of imaging modalities and conditions. In a public software benchmark competition, it outperformed all other fitters on 12 out of 12 datasets when comparing both detection accuracy and localization error, often by a substantial margin. DECODE allowed us to acquire fast dynamic live-cell SMLM data with reduced light exposure and to image microtubules at ultra-high labeling density. Packaged for simple installation and use, DECODE will enable many laboratories to reduce imaging times and increase localization density in SMLM., (© 2021. The Author(s), under exclusive licence to Springer Nature America, Inc.)

Unbiased quantitative analysis of macroscopic biological samples demands fast imaging systems capable of maintaining high resolution across large volumes. Here we introduce RAPID (rapid autofocusing via pupil-split image phase detection), a real-time autofocus method applicable in every widefield-based microscope. RAPID-enabled light-sheet microscopy reliably reconstructs intact, cleared mouse brains with subcellular resolution, and allowed us to characterize the three-dimensional (3D) spatial clustering of somatostatin-positive neurons in the whole encephalon, including densely labeled areas. Furthermore, it enabled 3D morphological analysis of microglia across the entire brain. Beyond light-sheet microscopy, we demonstrate that RAPID maintains high image quality in various settings, from in vivo fluorescence imaging to 3D tracking of fast-moving organisms. RAPID thus provides a flexible autofocus solution that is suitable for traditional automated microscopy tasks as well as for quantitative analysis of large biological specimens., (© 2021. The Author(s), under exclusive licence to Springer Nature America, Inc.)

Visualizing dynamic processes over large, three-dimensional fields of view at high speed is essential for many applications in the life sciences. Light-field microscopy (LFM) has emerged as a tool for fast volumetric image acquisition, but its effective throughput and widespread use in biology has been hampered by a computationally demanding and artifact-prone image reconstruction process. Here, we present a framework for artificial intelligence-enhanced microscopy, integrating a hybrid light-field light-sheet microscope and deep learning-based volume reconstruction. In our approach, concomitantly acquired, high-resolution two-dimensional light-sheet images continuously serve as training data and validation for the convolutional neural network reconstructing the raw LFM data during extended volumetric time-lapse imaging experiments. Our network delivers high-quality three-dimensional reconstructions at video-rate throughput, which can be further refined based on the high-resolution light-sheet images. We demonstrate the capabilities of our approach by imaging medaka heart dynamics and zebrafish neural activity with volumetric imaging rates up to 100 Hz.

Fast-paced innovations in imaging have resulted in single systems producing exponential amounts of data to be analyzed. Computational methods developed in computer science labs have proven to be crucial for analyzing these data in an unbiased and efficient manner, reaching a prominent role in most microscopy studies. Still, their use usually requires expertise in bioimage analysis, and their accessibility for life scientists has therefore become a bottleneck. Open-source software for bioimage analysis has developed to disseminate these computational methods to a wider audience, and to life scientists in particular. In recent years, the influence of many open-source tools has grown tremendously, helping tens of thousands of life scientists in the process. As creators of successful open-source bioimage analysis software, we here discuss the motivations that can initiate development of a new tool, the common challenges faced, and the characteristics required for achieving success., Competing Interests: No competing interests were disclosed., (Copyright: © 2021 Levet F et al.)

Emergence of the novel pathogenic coronavirus SARS-CoV-2 and its rapid pandemic spread presents challenges that demand immediate attention. Here, we describe the development of a semi-quantitative high-content microscopy-based assay for detection of three major classes (IgG, IgA, and IgM) of SARS-CoV-2 specific antibodies in human samples. The possibility to detect antibodies against the entire viral proteome together with a robust semi-automated image analysis workflow resulted in specific, sensitive and unbiased assay that complements the portfolio of SARS-CoV-2 serological assays. Sensitive, specific and quantitative serological assays are urgently needed for a better understanding of humoral immune response against the virus as a basis for developing public health strategies to control viral spread. The procedure described here has been used for clinical studies and provides a general framework for the application of quantitative high-throughput microscopy to rapidly develop serological assays for emerging virus infections., (© 2020 The Authors. BioEssays published by Wiley Periodicals LLC.)

We present ilastik, an easy-to-use interactive tool that brings machine-learning-based (bio)image analysis to end users without substantial computational expertise. It contains pre-defined workflows for image segmentation, object classification, counting and tracking. Users adapt the workflows to the problem at hand by interactively providing sparse training annotations for a nonlinear classifier. ilastik can process data in up to five dimensions (3D, time and number of channels). Its computational back end runs operations on-demand wherever possible, allowing for interactive prediction on data larger than RAM. Once the classifiers are trained, ilastik workflows can be applied to new data from the command line without further user interaction. We describe all ilastik workflows in detail, including three case studies and a discussion on the expected performance.

Segmentation is one of the most ubiquitous problems in biological image analysis. Here we present a machine learning-based solution to it as implemented in the open source ilastik toolkit. We give a broad description of the underlying theory and demonstrate two workflows: Pixel Classification and Autocontext. We illustrate their use on a challenging problem in electron microscopy image segmentation. After following this walk-through, we expect the readers to be able to apply the necessary steps to their own data and segment their images by either workflow.

Advances in the application of electron microscopy (EM) to serial imaging are opening doors to new ways of analyzing cellular structure. New and improved algorithms and workflows for manual and semiautomated segmentation allow us to observe the spatial arrangement of the smallest cellular features with unprecedented detail in full three-dimensions. From larger samples, higher complexity models can be generated; however, they pose new challenges to data management and analysis. Here we review some currently available solutions and present our approach in detail. We use the fully immersive virtual reality (VR) environment CAVE (cave automatic virtual environment), a room in which we are able to project a cellular reconstruction and visualize in 3D, to step into a world created with Blender, a free, fully customizable 3D modeling software with NeuroMorph plug-ins for visualization and analysis of EM preparations of brain tissue. Our workflow allows for full and fast reconstructions of volumes of brain neuropil using ilastik, a software tool for semiautomated segmentation of EM stacks. With this visualization environment, we can walk into the model containing neuronal and astrocytic processes to study the spatial distribution of glycogen granules, a major energy source that is selectively stored in astrocytes. The use of CAVE was key to the observation of a nonrandom distribution of glycogen, and led us to develop tools to quantitatively analyze glycogen clustering and proximity to other subcellular features., (© 2015 Wiley Periodicals, Inc.)

Tracking crowded cells or other targets in biology is often a challenging task due to poor signal-to-noise ratio, mutual occlusion, large displacements, little discernibility, and the ability of cells to divide. We here present an open source implementation of conservation tracking (Schiegg et al., IEEE international conference on computer vision (ICCV). IEEE, New York, pp 2928-2935, 2013) in the ilastik software framework. This robust tracking-by-assignment algorithm explicitly makes allowance for false positive detections, undersegmentation, and cell division. We give an overview over the underlying algorithm and parameters, and explain the use for a light sheet microscopy sequence of a Drosophila embryo. Equipped with this knowledge, users will be able to track targets of interest in their own data.

The development of realistic neuroanatomical models of peripheral nerves for simulation purposes requires the reconstruction of the morphology of the myelinated fibres in the nerve, including their nodes of Ranvier. Currently, this information has to be extracted by semimanual procedures, which severely limit the scalability of the experiments. In this contribution, we propose a supervised machine learning approach for the detailed reconstruction of the geometry of fibres inside a peripheral nerve based on its high-resolution serial section images. Learning from sparse expert annotations, the algorithm traces myelinated axons, even across the nodes of Ranvier. The latter are detected automatically. The approach is based on classifying the myelinated membranes in a supervised fashion, closing the membrane gaps by solving an assignment problem, and classifying the closed gaps for the nodes of Ranvier detection. The algorithm has been validated on two very different datasets: (i) rat vagus nerve subvolume, SBFSEM microscope, 200 × 200 × 200 nm resolution, (ii) rat sensory branch subvolume, confocal microscope, 384 × 384 × 800 nm resolution. For the first dataset, the algorithm correctly reconstructed 88% of the axons (241 out of 273) and achieved 92% accuracy on the task of Ranvier node detection. For the second dataset, the gap closing algorithm correctly closed 96.2% of the gaps, and 55% of axons were reconstructed correctly through the whole volume. On both datasets, training the algorithm on a small data subset and applying it to the full dataset takes a fraction of the time required by the currently used semiautomated protocols. Our software, raw data and ground truth annotations are available at http://hci.iwr.uni-heidelberg.de/Benchmarks/. The development version of the code can be found at https://github.com/RWalecki/ATMA., (© 2015 The Authors Journal of Microscopy © 2015 Royal Microscopical Society.)

This protocol describes how in vivo-imaged dendrites and axons in adult mouse brains can subsequently be prepared and imaged with focused ion beam scanning electron microscopy (FIBSEM). The procedure starts after in vivo imaging with chemical fixation, followed by the identification of the fluorescent structures of interest. Their position is then highlighted in the fixed tissue by burning fiducial marks with the two-photon laser. Once the section has been stained and resin-embedded, a small block is trimmed close to these marks. Serially aligned EM images are acquired through this region, using FIBSEM, and the neurites of interest are then reconstructed semiautomatically by using the ilastik software (http://ilastik.org/). This reliable imaging and reconstruction technique avoids the use of specific labels to identify the structures of interest in the electron microscope, enabling optimal chemical fixation techniques to be applied and providing the best possible structural preservation for 3D analysis. The entire protocol takes ∼4 d.