Search

For more than two centuries scientists and engineers have worked to understand and model how the eye encodes electromagnetic radiation (light). We now understand the principles of how light is transmitted through the optics of the eye and encoded by retinal photoreceptors and light-sensitive neurons. In recent years, new instrumentation has enabled scientists to measure the specific parameters of the optics and photoreceptor encoding. We implemented the principles and parameter estimates that characterize the human eye in an open-source software toolbox. This chapter describes the principles behind these tools and illustrates how to use them to compute the initial visual encoding.

Adaptive optics scanning laser ophthalmoscopy (AOSLO) is a high-resolution imaging modality that allows measurements of cellular-level retinal changes in living patients. In retinal diseases, the visibility of photoreceptors in AOSLO images is affected by pathology, patient motion, and optics, which can lead to variability in analyses of the photoreceptor mosaic. Current best practice for AOSLO mosaic quantification requires manual assessment of photoreceptor visibility across overlapping images, a laborious and time-consuming task.We propose an automated measure for quantification of photoreceptor visibility in AOSLO. Our method detects salient edge features, which can represent visible photoreceptor boundaries in each image. We evaluate our measure against two human graders and two standard automated image quality assessment algorithms.We evaluate the accuracy of pairwise ordering (PO) and the correlation of ordinal rankings (ORs) of photoreceptor visibility in 29 retinal regions, taken from five subjects with choroideremia. The proposed measure had high association with manual assessments (Grader 1: PO = 0.71, OR = 0.61; Grader 2: PO = 0.67, OR = 0.62), which is comparable with intergrader reliability (PO = 0.76, OR = 0.75) and outperforms the top standard approach (PO = 0.57; OR = 0.46).Our edge-based measure can automatically assess photoreceptor visibility and order overlapping images within AOSLO montages. This can significantly reduce the manual labor required to generate high-quality AOSLO montages and enables higher throughput for quantitative studies of photoreceptors.Automated assessment of photoreceptor visibility allows us to more rapidly quantify photoreceptor morphology in the living eye. This has applications to ophthalmic medicine by allowing detailed characterization of retinal degenerations, thus yielding potential biomarkers of treatment safety and efficacy.

Purpose Recent advances in retinal imaging allow visualization of structural abnormalities in retinal disease at the cellular level. This study used adaptive optics (AO) microperimetry to assess visual sensitivity with high spatial precision and to examine how function varies across 2 phenotypic features observed in choroideremia: atrophic lesion borders and outer retinal tubulations (ORTs). Design Cross-sectional study. Participants Twelve choroideremia patients. Methods A custom AO scanning light ophthalmoscope (AOSLO) equipped with both confocal and nonconfocal split-detection imaging methods was used to image the photoreceptor inner and outer segment mosaics. For AO microperimetry, circular 550-nm stimuli were presented through the AOSLO system; stimuli were either 9.6 or 38.3 arcmin2 (approximately 60 or 15 times smaller than a Goldman III stimulus). Test locations were identified in structural images and stimuli were targeted to these locations using real-time retinal tracking combined with measurements of transverse chromatic aberration. Psychophysical detection thresholds were measured at the atrophic border in 12 patients. Additionally, visual sensitivity was probed along ORTs in 4 patients. Main Outcome Measure Visual sensitivity thresholds measured with AO microperimetry at retinal locations corresponding to structural phenotypes observed on AOSLO retinal images. Results In choroideremia, sharp borders between intact central islands of the photoreceptor mosaic and complete atrophy of the outer retina and retinal pigment epithelium were observed in both split-detection and confocal structural images. Adaptive optics microperimetry at locations spanning these borders showed a commensurately sharp decrease in function, with readily measurable visual sensitivity on one side and dense scotoma on the other. These functional transitions often occurred over a distance smaller than the diameter of the Goldman III stimulus. Thresholds measured along ORTs showed dense scotoma over the tubule in all 4 participants, despite the visibility of remnant cone inner segments on the AO images. Conclusions Choroideremia patients exhibited sharp functional transitions that collocated with structural transitions from intact to severely degenerated retina. We found no evidence of visual sensitivity over ORTs. Measuring cone function with high resolution offered insight into disease mechanisms and may enable precise assessment of whether experimental therapies, such as gene therapy, provide a functional benefit.

Significance Do we talk about some colors more often than others? And do the colors we communicate about most frequently vary across cultures? A classic finding shows that languages around the world partition colors into words in remarkably similar, although not identical, ways. The biology of human color perception helps explain similar color vocabularies across languages, but less is known about how often speakers need to reference different colors. The inference method we develop reveals extensive variation in communicative needs across colors, and a diversity in needs across 130 languages, which helps explain variation in their color vocabularies. Our results open the door to studying cross-cultural variation in demands on different colors, and factors that drive color demands in linguistic communities., Names for colors vary widely across languages, but color categories are remarkably consistent. Shared mechanisms of color perception help explain consistent partitions of visible light into discrete color vocabularies. But the mappings from colors to words are not identical across languages, which may reflect communicative needs—how often speakers must refer to objects of different color. Here we quantify the communicative needs of colors in 130 different languages by developing an inference algorithm for this problem. We find that communicative needs are not uniform: Some regions of color space exhibit 30-fold greater demand for communication than other regions. The regions of greatest demand correlate with the colors of salient objects, including ripe fruits in primate diets. Our analysis also reveals a hidden diversity in the communicative needs of colors across different languages, which is partly explained by differences in geographic location and the local biogeography of linguistic communities. Accounting for language-specific, nonuniform communicative needs improves predictions for how a language maps colors to words, and how these mappings vary across languages. Our account closes an important gap in the compression theory of color naming, while opening directions to study cross-cultural variation in the need to communicate different colors and its impact on the cultural evolution of color categories.

An important goal for vision science is to develop quantitative models for the representation of visual signals at post-receptoral sites. To this end, we develop the quadratic color model (QCM) and examine its ability to account for the BOLD fMRI response in human V1 to spatially-uniform temporal chromatic modulations, systematically varying in their chromatic directions and contrasts. We find that the QCM explains the same, cross-validated variance as a conventional GLM, with far fewer free parameters. The QCM generalizes to allow prediction of V1 responses to a large range of modulations. We replicated the results for each subject and find good agreement across both replications and subjects. We find that within the LM cone contrast plane, V1 is most sensitive to L-M contrast modulations and least sensitive to L+M contrast modulations. Within V1, we observe little to no change in chromatic sensitivity as a function of eccentricity.

In addition to the cone photoreceptors the retina contains intrinsically photosensitive retinal ganglion cells (ipRGCs). These cells express the photopigment melanopsin and are known to be involved in reflexive visual functions such as pupil response and photo-entrainment of the circadian rhythm. It is possible that the ipRGCs contribute to conscious visual perception, either by providing an independent signal to the geniculo-striate pathway, or by interacting with and thus modifying signals arising from “classical” retinal ganglion cells that combine and contrast cone input. Here, we tested for the existence of an interaction by asking if a 350% change in melanopsin stimulation alters psychophysical sensitivity for the detection of luminance flicker. In Experiment 1, we tested for a change in the threshold for detecting luminance flicker in three participants after they adapted to backgrounds with different degrees of tonic melanopsin stimulation. In Experiments 2 and 3, this test was repeated, but now for luminance flicker presented on a transient pedestal of melanopsin stimulation. Across the three experiments, no effect of melanopsin stimulation upon threshold flicker sensitivity was found. Our results suggest that even large changes in melanopsin stimulation do not affect near-threshold, cone-mediated visual perception.

Textbook trichromacy accounts for human color vision in terms of spectral sampling by three classes of cone photoreceptors. This account neglects entangling of color and pattern information created by wavelength-dependent optical blur (chromatic aberrations) and interleaved spatial sampling of the retinal image by the three classes of cones. Recent experimental, computational, and neurophysiological work is now considering color and pattern vision at the elementary scale of daylight vison, that is at the scale of individual cones. The results provide insight about rich interactions between color and pattern vision as well as the role of the statistical structure of natural scenes in shaping visual processing.

We developed an image-computable observer model of the initial visual encoding that operates on natural image input, based on the framework of Bayesian image reconstruction from the excitations of the retinal cone mosaic. Our model extends previous work on ideal observer analysis and evaluation of performance beyond psychophysical discrimination, takes into account the statistical regularities of the visual environment, and provides a unifying framework for answering a wide range of questions regarding the visual front end. Using the error in the reconstructions as a metric, we analyzed variations of the number of different photoreceptor types on human retina as an optimal design problem. In addition, the reconstructions allow both visualization and quantification of information loss due to physiological optics and cone mosaic sampling, and how these vary with eccentricity. Furthermore, in simulations of color deficiencies and interferometric experiments, we found that the reconstructed images provide a reasonable proxy for modeling subjects’ percepts. Lastly, we used the reconstruction-based observer for the analysis of psychophysical threshold, and found notable interactions between spatial frequency and chromatic direction in the resulting spatial contrast sensitivity function. Our method is widely applicable to experiments and applications in which the initial visual encoding plays an important role.

We are sad to report that Professor Jacob (Jack) Nachmias passed away on March 2, 2019. Nachmias was born in Athens, Greece, on June 9, 1928. To escape the Nazis, he and his family came to the United States in 1939. He received his undergraduate degree from Cornell University and then an MA from Swarthmore College, where he worked with Hans Wallach and Wolfgang Kohler; his PhD in Psychology was from Harvard University. Nachmias spent the majority of his career as a Professor of Psychology at the University of Pennsylvania. He made fundamental contributions to our understanding of vision, most notably through the study of eye movements, the development of signal detection theory and forced-choice psychophysical methods, and the psychophysical characterization of spatial-frequency-selective visual channels. Nachmias' work was recognized by his election to the National Academy of Sciences and receipt of the Optical Society's Tillyer Award.

Background and ObjectivesTo quantify interictal photophobia in migraine with and without aura using reflexive eye closure as an implicit measure of light sensitivity and to assess the contribution of melanopsin and cone signals to these responses.MethodsParticipants were screened to meet criteria for 1 of 3 groups: headache-free (HF) controls, migraine without aura (MO), and migraine with visual aura (MA). MO and MA participants were included if they endorsed ictal and interictal photophobia. Exclusion criteria included impaired vision, inability to collect usable pupillometry, and history of either head trauma or seizure. Participants viewed light pulses that selectively targeted melanopsin, the cones, or their combination during recording of orbicularis oculi EMG (OO-EMG) and blinking activity.ResultsWe studied 20 participants in each group. MA and MO groups reported increased visual discomfort to light stimuli (discomfort rating, 400% contrast, MA: 4.84 [95% confidence interval 0.33, 9.35]; MO: 5.23 [0.96, 9.50]) as compared to HF controls (2.71 [0, 6.47]). Time course analysis of OO-EMG and blinking activity demonstrated that reflexive eye closure was tightly coupled to the light pulses. The MA group had greater OO-EMG and blinking activity in response to these stimuli (EMG activity, 400% contrast: 42.9%Δ [28.4, 57.4]; blink activity, 400% contrast: 11.2% [8.8, 13.6]) as compared to the MO (EMG activity, 400% contrast: 9.9%Δ [5.8, 14.0]; blink activity, 400% contrast: 4.7% [3.5, 5.9]) and HF control (EMG activity, 400% contrast: 13.2%Δ [7.1, 19.3]; blink activity, 400% contrast: 4.5% [3.1, 5.9]) groups.DiscussionOur findings suggest that the intrinsically photosensitive retinal ganglion cells (ipRGCs), which integrate melanopsin and cone signals, provide the afferent input for light-induced reflexive eye closure in a photophobic state. Moreover, we find a dissociation between implicit and explicit measures of interictal photophobia depending on a history of visual aura in migraine. This implies distinct pathophysiology in forms of migraine, interacting with separate neural pathways by which the amplification of ipRGC signals elicits implicit and explicit signs of visual discomfort.

Photoreceptors mediate the first step of vision, transducing light and passing signals to retinal neurons that ultimately send signals along the optic nerve to the brain. A functional deficiency in the photoreceptors, due to either congenital or acquired disease, can significantly affect an individual’s sight and quality of life. Methods for quantifying the health and function of photoreceptors are essential for understanding both the progression of disease and the efficacy of treatment. Given that emerging treatments such as gene, stem cell, and small molecule therapy are designed to operate at the cellular scale, it is desirable to monitor function at the commensurate resolution of individual photoreceptors. Previously, non-invasive imaging methods for visualizing photoreceptor mosaic structure have been used to infer photoreceptor health, but these methods do not assess function directly. Conversely, most functional techniques, such as ERG and conventional microperimetry, measure function by aggregating the effects of signals from many photoreceptors. We have previously shown that stimulus-evoked intrinsic changes in intensity can be measured reliably in populations of cone photoreceptors in the intact human eye, a measurement we refer to more generally as the cone optoretinogram. Here we report that we can resolve the intensity optoretinogram at the level of individual cones. Moreover, we show that the individual cone optoretinogram exhibits two key signatures expected of a functional measure. First, responses in individual cones increase systematically as a function of stimulus irradiance. Second, we can use the amplitude of the functional response to middle wavelength (545 nm) light to separate the population of short-wavelength-sensitive (S) cones from the population of middle- and long-wavelength-sensitive (L and M) cones. Our results demonstrate the promise of optoretinography as a direct diagnostic measure of individual cone function in the living human eye.

ObjectiveTo assess the contribution of the melanopsin-containing, intrinsically photosensitive retinal ganglion cells (ipRGCs) and the cones to reflexive eye closure as an implicit measure of interictal photophobia in migraine.MethodsWe studied twenty participants in each of three groups: headache-free (HAf) controls, migraine without aura (MwoA), and migraine with visual aura (MwA). Participants viewed spectral pulses that selectively targeted melanopsin, the cones, or their combination while we recorded orbicularis oculi electromyography (OO-EMG) and blinking rate.ResultsTime course analysis of OO-EMG demonstrated that reflexive eye closure was tightly coupled to the spectral pulses. Compared to both the MwoA and HAf control groups, the MwA group had enhanced OO-EMG and blinking activity in response to melanopsin and cone stimulation in combination and in isolation. This response scaled with the contrast of the stimulus.ConclusionsOur findings suggest that ipRGC signals, whether elicited by melanopsin stimulation or from presumed extrinsic cone input, provide the afferent input for light-induced reflexive eye closure in a photophobic state. Participants with migraine and visual aura had a distinctly different response to visual stimulation as compared to the other two groups. This is in contrast to prior findings in this same cohort in whom higher explicit ratings of visual discomfort were found for both MwA and MwoA as compared to controls. Such a dissociation suggests distinct pathophysiology in forms of migraine, interacting with separate neural pathways by which ipRGC signals elicit implicit and explicit signs of visual discomfort.

We have recently shown that using the information carried by the mosaic of cone excitations of a stationary retina, the relative spatial contrast sensitivity function (CSF) of a computational observer has the same shape as a typical human subject. Absolute human sensitivity, however, is lower than the computational observer by a factor of 5 to 10. Here we model how additional known features of early vision affect spatial contrast sensitivity: fixational eye movements and the conversion of cone photopigment excitations to cone photocurrent responses. For a computational observer that uses a linear classifier applied to the responses of a stimulus-matched linear filter, fixational eye movements substantially change the shape of the spatial CSF, primarily by reducing sensitivity at spatial frequencies above 10 c/deg. For a computational observer that uses a translation-invariant calculation, in which decisions are based on the squared response of a quadrature-pair of linear filters, the CSF shape is little changed by eye movements, but there is a two-fold reduction in sensitivity. The noise and response dynamics of conversion of cone excitations into photocurrent introduce an additional two-fold sensitivity decrease. Hence, the combined effects of fixational eye movements and phototransduction bring the absolute sensitivity of the translation-invariant computational observer CSF to within a factor of 1 to 2 of the human CSF. We note that the human CSF depends on processing of the initial representation by many thalamic and cortical neurons, which are individually quite noisy. Our computational modeling suggests that the net effect of this noise on contrast-detection performance, when considered at the neural population level and behavioral level, is quite small: the inference mechanisms that determine the CSF, presumably in cortex, make efficient use of the information available from the cone photocurrents of the fixating eye.

Second only to headache, photophobia is the most debilitating symptom reported by people with migraine. While the melanopsin-containing, intrinsically photosensitive retinal ganglion cells (ipRGCs) are thought to play a role, how cone and melanopsin signals are integrated in this pathway to produce visual discomfort is poorly understood.We studied 60 people: 20 without headache and 20 each with interictal photophobia from migraine with or without aura. Participants viewed pulses of spectral change that selectively targeted melanopsin, the cones, or both, and rated the degree of visual discomfort produced by these stimuli while we also recorded pupil responses.We examined the data within a model that describes how cone and melanopsin signals are weighted and combined at the level of the retina, and how this combined signal is transformed into a rating of discomfort or pupil response. Our results indicate that people with migraine do not differ from headache-free controls in the manner in which melanopsin and cone signals are combined. Instead, people with migraine demonstrate an amplification of integrated ipRGC signals for discomfort. This effect of migraine is selective for ratings of visual discomfort, in that an amplification of pupil responses was not seen in the migraine group, nor were group differences found in surveys of other behaviors putatively linked to ipRGC function (chronotype, seasonal sensitivity, presence of a photic sneeze reflex).By revealing a dissociation in the amplification of discomfort versus pupil response, our findings suggest a post-retinal alteration in processing of ipRGC signals for photophobia in migraine.SignificanceThe melanopsin-containing, intrinsically photosensitive retinal ganglion cells (ipRGCs) may contribute to photophobia in migraine. We measured visual discomfort and pupil responses to cone and melanopsin stimulation—the photoreceptor inputs to the ipRGCs—in people with and without migraine. We find that people with migraine do not differ from those without headaches in how cone and melanopsin signals are weighted and combined to produce visual discomfort. Instead, migraine is associated with an amplification of ipRGC signals for discomfort. This effect of migraine upon ipRGC signals is limited to photophobia, as we did not find an enhancement of pupil responses or a change in other behaviors linked to ipRGC function. Our findings suggest a post-retinal amplification of ipRGC signals for photophobia in migraine.

We use adaptive optics scanning laser ophthalmoscopy to measure the cone optoretinogram, an optical measure of photoreceptor function that arises from a change in nearinfrared reflectance in response to visual stimuli.