Search

Understanding the spatial organization of tissues is of critical importance for both basic and translational research. While recent advances in tissue imaging are opening an exciting new window into the biology of human tissues, interpreting the data that they create is a significant computational challenge. Cell segmentation, the task of uniquely identifying each cell in an image, remains a substantial barrier for tissue imaging, as existing approaches are inaccurate or require a substantial amount of manual curation to yield useful results. Here, we addressed the problem of cell segmentation in tissue imaging data through large-scale data annotation and deep learning. We constructed TissueNet, an image dataset containing >1 million paired whole-cell and nuclear annotations for tissue images from nine organs and six imaging platforms. We created Mesmer, a deep learning-enabled segmentation algorithm trained on TissueNet that performs nuclear and whole-cell segmentation in tissue imaging data. We demonstrated that Mesmer has better speed and accuracy than previous methods, generalizes to the full diversity of tissue types and imaging platforms in TissueNet, and achieves human-level performance for whole-cell segmentation. Mesmer enabled the automated extraction of key cellular features, such as subcellular localization of protein signal, which was challenging with previous approaches. We further showed that Mesmer could be adapted to harness cell lineage information present in highly multiplexed datasets. We used this enhanced version to quantify cell morphology changes during human gestation. All underlying code and models are released with permissive licenses as a community resource.

Live-cell imaging experiments have opened an exciting window into the behavior of living systems. While these experiments can produce rich data, the computational analysis of these datasets is challenging. Single-cell analysis requires that cells be accurately identified in each image and subsequently tracked over time. Increasingly, deep learning is being used to interpret microscopy image with single cell resolution. In this work, we apply deep learning to the problem of tracking single cells in live-cell imaging data. Using crowdsourcing and a human-in-the-loop approach to data annotation, we constructed a dataset of over 11,000 trajectories of cell nuclei that includes lineage information. Using this dataset, we successfully trained a deep learning model to perform cell tracking within a linear programming framework. Benchmarking tests demonstrate that our method achieves state-of-the-art performance on the task of cell tracking with respect to multiple accuracy metrics. Further, we show that our deep learning-based method generalizes to perform cell tracking for both fluorescent and brightfield images of the cell cytoplasm, despite having never been trained on those data types. This enables analysis of live-cell imaging data collected across imaging modalities. A persistent cloud deployment of our cell tracker is available athttp://www.deepcell.org.

A bipolar (BP) nanosecond electric pulse (nsEP) exposure generates reduced calcium influx compared to a unipolar (UP) nsEP. This attenuated physiological response from a BP nsEP exposure is termed “bipolar cancellation” (BPC). The predominant BP nsEP parameters that induce BPC consist of a positive polarity (↑) front pulse followed by the delivery of a negative polarity (↓) back pulse of equal voltage and width; thereby the duration is twice a UP nsEP exposure. We tested these BPC parameters, and discovered that a BP nsEP with symmetrical pulse widths is not required to generate BPC. For example, our data revealed the physiological response initiated by a ↑900 nsEP exposure can be cancelled by a second pulse that is a third of its duration. However, we observed a complete loss of BPC from a ↑300 nsEP followed by a ↓900 nsEP exposure. Spatiotemporal analysis revealed these asymmetrical BP nsEP exposures generate distinct local YO-PRO®-1 uptake patterns across the plasma membrane. From these findings, we generated a conceptual model that suggests BPC is a phenomenon balanced by localized charging and discharging events across the membrane.

Recent advances in computer vision and machine learning underpin a collection of algorithms with an impressive ability to decipher the content of images. These deep learning algorithms are being applied to biological images and are transforming the analysis and interpretation of imaging data. These advances are positioned to render difficult analyses routine and to enable researchers to carry out new, previously impossible experiments. Here we review the intersection between deep learning and cellular image analysis and provide an overview of both the mathematical mechanics and the programming frameworks of deep learning that are pertinent to life scientists. We survey the field's progress in four key applications: image classification, image segmentation, object tracking, and augmented microscopy. Last, we relay our labs' experience with three key aspects of implementing deep learning in the laboratory: annotating training data, selecting and training a range of neural network architectures, and deploying solutions. We also highlight existing datasets and implementations for each surveyed application.

The Game-Engine-Assisted Research platform for Scientific computing (GEARS) is a visualization framework developed at the Materials Genome Innovation for Computational Software (MAGICS) center to perform simulations and on-the-fly data exploration in virtual reality (VR) environments. This hardware-agnostic framework accommodates multiple programming languages and game engines in addition to supporting integration with a widely-used materials simulation engine called LAMMPS. GEARS also features a novel data exploration tool called virtual confocal microscopy, which endows scientific visualization with enhanced functionality. Keywords: Virtual reality, Real-time simulation visualization, Molecular dynamics, LAMMPS

Recently, we presented a computational framework named VizBET to simulate and visualize biological electron-transfer (ET) dynamics. The visualization process was encapsulated as a plugin to the Visual Molecular Dynamics (VMD) software. However, the user's ability to understand complex, multidimensional ET pathways was severely limited when visualized in 2D on traditional computer monitors. To provide a more accurate representation with enhanced depth perception, we here present an extension of VizBET named iBET to render the VMD model of ET dynamics in a commodity virtual reality (VR) platform. The paper describes detailed procedures to export VMD models into the Unity game engine and render it in an Oculus Rift head mounted display. With the increasing availability of low-cost VR systems like the Rift and rich programmability of game engines, the iBET framework provides a powerful means to explore and understand not only biological ET processes but also a unique experiential tool for broad scientific communities.

Nanosecond electric pulse (nsEP) exposure generates an array of physiological effects. The extent of these effects is impacted by whether the nsEP is a unipolar (UP) or bipolar (BP) exposure. A 600 ns pulse can generate 71% more YO-PRO-1 uptake compared to a 600 ns + 600 ns pulse exposure. This observation is termed "bipolar cancellation" (BPC) because despite the BP nsEP consisting of an additional 600 ns pulse, it generates reduced membrane perturbation. BPC is achieved by varying pulse amplitudes, and symmetrical and asymmetric pulse widths. The effect appears to reverse by increasing the interphase interval between symmetric BP pulses, suggesting membrane recovery is a BPC factor. To date, the impact of the interphase interval between asymmetrical BP and other BPC-inducing symmetrical BP nsEPs has not been fully explored. Additionally, interpulse intervals beyond 50 μs have not been explored to understand the impact of time between the BP nsEP phases. Here, we surveyed different interphase intervals among symmetrical and asymmetrical BP nsEPs to monitor their impact on BPC of YO-PRO-1 uptake. We identified that a 10 microsecond (ms) interphase interval within a symmetrical 600 ns + 600 ns, and 900 ns + 900 ns pulse can resolve BPC. Furthermore, the interphase interval to resolve asymmetric BPC from a 300 ns + 900 ns pulse versus 600 ns pulse exposure is greater (

Electric-field induced physical phenomena, such as thermal, mechanical and electrochemical dynamics, may be the driving mechanism behind bioeffects observed in mammalian cells during exposure to nanosecond-duration electric pulses (nsEP) in-vitro. Correlating a driving mechanism to a biological response requires the experimental measurement and quantification of all physical dynamics resulting from the nsEP stimulus. A passive and electromagnetic interference (EMI) immune sensor is required to resolve these dynamics in high strength electric fields. The probe beam deflection technique (PBDT) is a passive and EMI immune optical method for quantifying and imaging refractive index gradients in liquids and gases, both dynamic and static, with nanosecond temporal resolution. In this work, a probe beam deflection imaging system was designed to acquire 2-D time-lapse images of thermal/mechanical dynamics resulting from monopolar and bipolar nsEP stimulus.

Nanosecond pulsed electric fields (nsPEF) have proven useful for transporting cargo across cell membranes and selectively activating cellular pathways. The chemistry and biophysics governing this cellular response, however, are complex and not well understood. Recent studies have shown that the conductivity of the solution cells are exposed in could play a significant role in plasma membrane permeabilization and, thus, the overall cellular response. Unfortunately, the means of detecting this membrane perturbation has traditionally been limited to analyzing one possible consequence of the exposure – diffusion of molecules across the membrane. This method has led to contradictory results with respect to the relationship between permeabilization and conductivity. Diffusion experiments also suffer from “saturation conditions” making multi-pulse experiments difficult. As a result, this method has been identified as a key stumbling block to understanding the effects of nsPEF exposure. To overcome these limitations, we recently developed a nonlinear optical imaging technique based on second harmonic generation (SHG) that allows us to identify nanoporation in live cells during the pulse in a wide array of conditions. As a result, we are able to explore and fully test whether lower conductivity extracellular solutions could induce more efficient nanoporation. This hypothesis is based on membrane charging and the relative difference between the extracellular solution and the cytoplasm. The experiments also allow us to test the noise floor of our methodology against the effects of ion leakage. The results emphasize that the electric field, not ionic phenomenon, are the driving force behind nsPEF-induced membrane nanoporation.

We recently developed a nonlinear optical imaging technique based on second harmonic generation (SHG) to identify membrane disruption events in live cells. This technique was used to detect nanoporation in the plasma membrane following nanosecond pulsed electric field (nsPEF) exposure. It has been hypothesized that similar poration events could be induced by the thermal gradients generated by infrared (IR) laser energy. Optical pulses are a highly desirable stimulus for the nervous system, as they are capable of inhibiting and producing action potentials in a highly localized but non-contact fashion. However, the underlying mechanisms involved with infrared neural stimulation (INS) are not well understood. The ability of our method to non-invasively measure membrane structure and transmembrane potential via Two Photon Fluorescence (TPF) make it uniquely suited to neurological research. In this work, we leverage our technique to understand what role membrane structure plays during INS and contrast it with nsPEF stimulation. We begin by examining the effect of IR pulses on CHO-K1 cells before progressing to primary hippocampal neurons. The use of these two cell lines allows us to directly compare poration as a result of IR pulses to nsPEF exposure in both a neuron-derived cell line, and one likely lacking native channels sensitive to thermal stimuli.

Plasma membrane disruption can trigger a host of cellular activities. One commonly observed type of disruption is pore formation. Molecular dynamic (MD) simulations of simplified lipid membrane structures predict that controllably disrupting the membrane via nano-scale poration may be possible with nanosecond pulsed electric fields (nsPEF). Until recently, researchers hoping to verify this hypothesis experimentally have been limited to measuring the relatively slow process of fluorescent markers diffusing across the membrane, which is indirect evidence of nanoporation that could be channel-mediated. Leveraging recent advances in nonlinear optical microscopy, we elucidate the role of pulse parameters in nsPEF-induced membrane permeabilization in live cells. Unlike previous techniques, it is able to directly observe loss of membrane order at the onset of the pulse. We also develop a complementary theoretical model that relates increasing membrane permeabilization to membrane pore density. Due to the significantly improved spatial and temporal resolution possible with our imaging method, we are able to directly compare our experimental and theoretical results. Their agreement provides substantial evidence that nanoporation does occur and that its development is dictated by the electric field distribution.

Nanosecond electrical pulse (nsEP) exposure activates signaling pathways, produces oxidative stress, stimulates hormone secretion, causes cell swelling and induces apoptotic and necrotic death. The underlying biophysical connection(s) between these diverse cellular reactions and nsEP has yet to be elucidated. Using global genetic analysis, we evaluated how two commonly studied cell types, U937 and Jurkat, respond to nsEP exposure. We hypothesized that by studying the genetic response of the cells following exposure, we would gain direct insight into the stresses experienced by the cell and in turn better understand the biophysical interaction taking place during the exposure. Using Ingenuity Systems software, we found genes associated with cell growth, movement and development to be significantly up-regulated in both cell types 4 h post exposure to nsEP. In agreement with our hypothesis, we also found that both cell lines exhibit significant biological changes consistent with mechanical stress induction. These results advance nsEP research by providing strong evidence that the interaction of nsEPs with cells involves mechanical stress.

Second Harmonic Generation (SHG) imaging is a useful tool for examining the structure of interfaces between bulk materials. Recently, this technique was applied to detecting subtle perturbations in the structure of cellular membranes following nanosecond pulsed electric field (nsPEF) exposure. Monitoring the cell’s outer membrane as it is exposed to nsPEF via SHG has demonstrated that nanoporation is likely the root cause for size-specific, increased cytoplasmic membrane permeabilization. It is theorized that the area of the membrane covered by these pores is tied to pulse intensity or duration. The extent of this effect along the cell’s surface, however, has never been measured due to its temporal brevity and minute pore size. By enhancing the SHG technique developed and elucidated previously, we are able to obtain this information. Further, we vary the pulse width and amplitude of the applied stimulus to explore the mechanical changes of the membrane at various sites around the cell. By using this unique SHG imaging technique to directly visualize the change in order of phospholipids within the membrane, we are able to better understand the complex response of living cells to electric pulses.

The requirement of center asymmetry for the creation of second harmonic generation (SHG) signals makes it an attractive technique for visualizing changes in interfacial layers such as the plasma membrane of biological cells. In this article, we explore the use of lipophilic SHG probes to detect minute perturbations in the plasma membrane. Three candidate probes, Di-4-ANEPPDHQ (Di-4), FM4-64, and all-trans-retinol, were evaluated for SHG effectiveness in Jurkat cells. Di-4 proved superior with both strong SHG signal and limited bleaching artifacts. To test whether rapid changes in membrane symmetry could be detected using SHG, we exposed cells to nanosecond-pulsed electric fields, which are believed to cause formation of nanopores in the plasma membrane. Upon nanosecond-pulsed electric fields exposure, we observed an instantaneous drop of ∼50% in SHG signal from the anodic pole of the cell. When compared to the simultaneously acquired fluorescence signals, it appears that the signal change was not due to the probe diffusing out of the membrane or changes in membrane potential or fluidity. We hypothesize that this loss in SHG signal is due to disruption in the interfacial nature of the membrane. The results show that SHG imaging has great potential as a tool for measuring rapid and subtle plasma membrane disturbance in living cells.

Nonlinear optical probes, especially those involving second harmonic generation (SHG), have proven useful as sensors for near-instantaneous detection of alterations to or ientation or energetics within a substance. This has been exploited to some success for observing conformational changes in proteins. SHG probes, therefore, hold promise for reporting rapid and minute changes in lipid membranes. In this report, one of these probes is employed in this regard, using nanosecond electric pulses (nsEPs) as a vehicle for instigating subtle membrane perturbations. The result provides a useful tool and methodology for the observation of minute membrane perturbation, while also providing meaningful information on the phenomenon of electropermeabilization due to nsEP. The SHG probe Di-4-ANEPPDHQ is used in conjunction with a tuned optical setup to demonstrate nanoporation preferential to one hemisphere, or pole, of the cell given a single square shaped pulse. The results also confirm a correlation of pulse width to the amount of poration. Furthermore, the polarity of this event and the membrane physics of both hemispheres, the poles facing either electrode, were tested using bipolar pulses consisting of two pulses of opposite polarity. The experiment corroborates findings by other researchers that these types of pulses are less effective in causing repairable damage to the lipid membrane of cells.

Second Harmonic Generation (SHG) microscopy is an effective tool for studying the order of symmetry-breaking interfaces, such as lipid bilayers. Recently, we have shown that disruptions in the interfacial nature of the membrane can be studied with the addition of lipophilic SHG probes, specifically Di-4-ANEPPDHQ (Di-4). In addition to the conformational information provided by the SHG signal, this particular probe can be used to determine lipid phase and transmembrane voltage through the dye's fluorescence behavior. We now strengthen our technique by changing the polarity of the incident laser beam from linear to circular polarization. The modification provides SHG signal around the circumference of the cell in every optical slice, while maintaining a sufficient signal-to-noise ratio. Utilizing nanosecond pulsed electric fields (nsPEFs) of varying pulse widths and intensities as a tool for provoking spatially and temporally minute perturbations in the cell membrane, we observe membrane poration on the nano-scale. To verify our results, we adapted a previously-published electrical membrane model to map the theoretical angular area affected by these nsPEFs. The tests further establish the effectiveness of SHG probes, in our case Di-4, as a tool for observing subtle membrane alterations and document the extent to which electric fields of varying pulse parameters affect the cellular membrane.

Electric pulses have become an effective tool for transporting cargo (DNA, drugs, etc.) across cell membranes. This enhanced transport is believed to occur through temporary pores formed in the plasma membrane. Traditionally, millisecond duration, monopolar (MP) pulses are used for electroporation, but bipolar (BP) pulses have proven equally effective as MP pulses with the added advantage of less cytotoxicity. With the goal of further reducing cytotoxic effects and inducing non-thermal, intra-cellular effects, researchers began investigating reduced pulse durations, pushing into the nanosecond regime. Cells exposed to these MP, nanosecond pulsed electric fields (nsPEFs) have shown increased repairable membrane permeability and selective channel activation. However, attempts to improve this further by moving to the BP pulse regime has proven unsuccessful. In the present work, we use second harmonic generation imaging to explore the structural effects of bipolar nsPEFs on the plasma membrane. By varying the...

Cells exposed to monopolar (MP) nanosecond pulsed electric fields (nsPEF) exhibit increased membrane permeability. Depending on the intensity and duration of the pulse, this damage could be repairable or induce cell death. Recently, we leveraged a new second harmonic generation (SHG) microscopy technique to study the underlying biophysical effects these pulses have on the cellular membrane. We observed nano-scale poration across the entire plasma membrane, but concentrated at the anodic-facing pole of the cell. The polarity of this event suggests that the resting membrane potential may be playing a role. To investigate further, we apply a symmetric bipolar (BP) nsPEF, where the polarity of the pulse is reversed halfway through the pulse duration. Previous studies have shown that BP nsPEF are less effective at inducing cell death than their MP counterparts. Our results suggest that this is due, at least in part, to the dramatically reduced amount of nanoporation present upon onset of BP nsPEF as compared to MP. If an interstitial “off” period is introduced between the two MP pulses making up the BP pulse, the amount of nanoporation increases commensurate with the space between the two MP pulses. This indicates that membrane charging is playing a large role in the nanoporation process.