Search

Multiplex immunofluorescence (mIF) assays multiple protein biomarkers on a single tissue section. Recently, high-plex CODEX (co-detection by indexing) systems enable simultaneous imaging of 40+ protein biomarkers, unlocking more detailed molecular phenotyping, leading to richer insights into cellular interactions and disease. However, high-plex imaging can be slower and more costly to collect, limiting its applications, especially in clinical settings. We propose a machine learning framework, 7-UP, that can computationally generate in silico 40-plex CODEX at single-cell resolution from a standard 7-plex mIF panel by leveraging cellular morphology. We demonstrate the usefulness of the imputed biomarkers in accurately classifying cell types and predicting patient survival outcomes. Furthermore, 7-UP’s imputations generalize well across samples from different clinical sites and cancer types. 7-UP opens the possibility of in silico CODEX, making insights from high-plex mIF more widely available.

Visual information arriving at the retina is transmitted to the brain by signals in the optic nerve, and the brain must rely solely on these signals to make inferences about the visual world. Previous work has probed the content of these signals by directly reconstructing images from retinal activity using linear regression or nonlinear regression with neural networks. Maximuma posteriori(MAP) reconstruction using retinal encoding models and separately-trained natural image priors offers a more general and principled approach. We develop a novel method for approximate MAP reconstruction that combines a generalized linear model for retinal responses to light, including their dependence on spike history and spikes of neighboring cells, with the image prior implicitly embedded in a deep convolutional neural network trained for image denoising. We use this method to reconstruct natural images fromex vivosimultaneously-recorded spikes of hundreds of retinal ganglion cells uniformly sampling a region of the retina. The method produces reconstructions that match or exceed the state-of-the-art in perceptual similarity and exhibit additional fine detail, while using substantially fewer model parameters than previous approaches. The use of more rudimentary encoding models (a linear-nonlinear-Poisson cascade) or image priors (a 1/fspectral model) significantly reduces reconstruction performance, indicating the essential role of both components in achieving high-quality reconstructed images from the retinal signal.

Multiplexed immunofluorescence imaging enables high-dimensional molecular profiling at subcellular resolution. However, learning disease-relevant cellular environments from these rich imaging data is an open challenge. We developed SPAtial CEllular Graphical Modeling (SPACE-GM), a geometric deep learning framework that flexibly models tumor microenvironments (TMEs) as cellular graphs. We applied SPACE-GM to 658 head-and-neck and colorectal human cancer samples assayed with 40-plex immunofluorescence imaging to identify spatial motifs associated with cancer recurrence and patient survival after immunotherapy. SPACE-GM is substantially more accurate in predicting patient outcomes than previous approaches for modeling spatial data using neighborhood cell-type compositions. Computational interpretation of the disease-relevant microenvironments identified by SPACE-GM generates insights into the effect of spatial dispersion of tumor cells and granulocytes on patient prognosis.

Electrical stimulation of retinal ganglion cells (RGCs), which transmit visual information to the brain, is used in retinal implants to treat blindness caused by photoreceptor degeneration. However, the performance of existing clinical implants is limited by indiscriminate stimulation of many cells and cell types. Recent work in isolated macaque retina has shown the ability to precisely evoke spikes in the major RGC types by direct electrical stimulation at safe current levels, with single-cell, single-spike resolution and avoidance of axon bundle activation in many cases. However, these findings have not been verified in the human retina. Here, electrical activation of the major human RGC types was examined using large-scale, multi-electrode recording and stimulation and compared to results from several macaque retinas obtained using the same methods. Electrical stimulation of the major human RGC types closely paralleled results in macaque, with similar somatic and axonal stimulation thresholds, cellular and cell type selectivity of stimulation, avoidance of axon bundle stimulation by calibration, targeting of different cell types based on their distinct electrical signatures, and potential efficacy of real-time stimulus optimization for artificial vision. The results indicate that the macaque retina provides a quantitatively accurate picture of how focal electrical stimulation can be used in future high-resolution implants.

The functional organization of diverse retinal ganglion cell (RGC) types, which shapes the visual signal transmitted to the brain, has been examined in many species. The unique spatial, temporal, and chromatic properties of the numerically dominant RGC types in macaque monkey retina are presumed to most accurately model human vision. However, the functional similarity between RGCs in macaques and humans has only begun to be tested, and recent work suggests possible differences. Here, the properties of the numerically dominant human RGC types were examined using large-scale multi-electrode recordings with fine-grained visual stimulation in isolated retina, and compared to results from dozens of recordings from macaque retina using the same experimental methods and conditions. The properties of four major human RGC types -- ON-parasol, OFF-parasol, ON-midget, and OFF-midget -- closely paralleled those of the same macaque RGC types, including the spatial and temporal light sensitivity, precisely coordinated mosaic organization of receptive fields, ON-OFF asymmetries, spatial response nonlinearity, and sampling of photoreceptor inputs over space. Putative smooth monostratified cells and polyaxonal amacrine cells were also identified based on similarities to cell types previously identified in macaque retina. The results suggest that recently proposed differences between human and macaque RGCs probably reflect experimental differences, and that the macaque model provides an accurate picture of human RGC function.