Search

Foveation has two methods: single image mode and dual images mode. For the former, SOC transmits the compressed image consists of different low‐resolution blocks, some of which have very large differences after up‐scaling may cause flickering risks when displayed. For the latter, SOC processes the compressed periphery and foveated images sequentially, therefor DDIC uses large GRAM to cache these bitstreams, which will also introduce huge latency. In order to solve these problems existing in two modes, this paper presents a scheme to process the image or compressed bitstreams with coding unit rearrangement. [ABSTRACT FROM AUTHOR]

We introduce the design of statically foveated head‐mounted displays based on a new perceptual‐driven approach, aiming to offer a large field of view and high resolution with imperceptible image quality degradation eliminating the need for eye‐tracking or scanning mechanisms. The system incorporates a uniform‐resolution display panel and a spatially varying optical power eyepiece. [ABSTRACT FROM AUTHOR]

Purpose: To describe the mechanisms of postoperative foveal restoration in three patients with bacillary layer detachment (BALAD) associated with macula-off rhegmatogenous retinal detachment. Observations: BALAD associated with macula-off rhegmatogenous retinal detachment presented with two different morphologies: with an intact foveal roof (case 1) and lamellar, with an open foveal roof (cases 2 and 3). In case 1 visual acuity significantly improved and foveal morphology completely restored at postoperative month 6, with a marked increase in foveal thickness. Case 2 presented with a lamellar BALAD in a long-standing retinal detachment, and it was treated with scleral buckling with an unfavourable evolution into a full-thickness hole in the early postoperative period. It was then addressed with internal limiting membrane peeling and inverted flap, which resulted in the resolution of the lesion, but with limited postoperative visual and anatomical recovery. Case 3 lamellar BALAD was directly addressed with pars plana vitrectomy, ILM peeling and inverted flap, with a remarkable foveal anatomical restoration and visual acuity improvement over the follow-up period. Conclusions and importance: The process of foveation may play a key role in the healing process of BALAD associated with rhegmatogenous retinal detachment. Lamellar BALAD should be considered and treated as a FTMH associated with retinal detachment. The two BALAD subtypes may represent different clinical stages of the BALAD spectrum.

Foveation can be defined as the organic action of directing the gaze towards a visual region of interest to acquire relevant information selectively. With the recent advent of event cameras, we believe that taking advantage of this visual neuroscience mechanism would greatly improve the efficiency of event data processing. Indeed, applying foveation to event data would allow to comprehend the visual scene while significantly reducing the amount of raw data to handle. In this respect, we demonstrate the stakes of neuromorphic foveation theoretically and empirically across several computer vision tasks, namely semantic segmentation and classification. We show that foveated event data have a significantly better trade-off between quantity and quality of the information conveyed than high- or low-resolution event data. Furthermore, this compromise extends even over fragmented datasets. Our code is publicly available online at: https://github.com/amygruel/FoveationStakes_DVS. [ABSTRACT FROM AUTHOR]

Video compression has become increasingly crucial as video resolution and bitrate have surged in recent years. However, most widely applied video compression methods do not fully exploit the characteristics of the Human Visual System (HVS) to reduce perceptual redundancy in videos. In this paper, we propose a novel video compression method that integrates visual saliency information with foveation to reduce perceptual redundancy. We present a new approach to subsample and restore the input image using saliency data, which allocates more space for salient regions and less for non-salient ones. We analyze the information entropy in video frames before and after applying our algorithm and demonstrate that the proposed method reduces redundancy. Through subjective and objective evaluations, we show that our method produces videos with superior perceptual visual quality. Moreover, our approach can be added to most existing video compression standards without altering their bitstream format.

With the rapid development of devices for virtual reality, massive amounts of multimedia data have been produced to enrich the immersive experience. To construct vivid virtual reality, high-resolution content is required to achieve a good user experience. With the limitations of hardware capability, efficient data compression is one of the keys to addressing the current dilemma. In our opinion, visual perception is a promising strategy for addressing this problem. In this study, the impact of human visual perception in immersive environment is studied. Two hypotheses accounting for their impact on the quality of immersive videos at a fixed viewport are proposed and validated by subjective evaluation and analysis. A series of videos are created with various blur distortions based on the stimulus triggered by depth perception and the rapid falloff of acuity in the peripheral vision. Stereoscopic videos with imposed blur distortions arranged with different severities are evaluated subjectively in an immersive viewing environment. We then follow statistical approaches to examine the collected subjective quality distributions to discuss the feasibility of video compression based on processed videos. To improve encoding efficiency, a heuristic mechanism for immersive visual content compression is proposed. We examine a simple compression framework to demonstrate that the proposed hypotheses are applicable for practical video compression.

In Virtual Reality (VR), the requirements of much higher resolution and smooth viewing experiences under rapid and often real-time changes in viewing direction, leads to significant challenges in compression and communication. To reduce the stresses of very high bandwidth consumption, the concept of foveated video compression is being accorded renewed interest. By exploiting the space-variant property of retinal visual acuity, foveation has the potential to substantially reduce video resolution in the visual periphery, with hardly noticeable perceptual quality degradations. Accordingly, foveated image / video quality predictors are also becoming increasingly important, as a practical way to monitor and control future foveated compression algorithms. Towards advancing the development of foveated image / video quality assessment (FIQA / FVQA) algorithms, we have constructed 2D and (stereoscopic) 3D VR databases of foveated / compressed videos, and conducted a human study of perceptual quality on each database. Each database includes 10 reference videos and 180 foveated videos, which were processed by 3 levels of foveation on the reference videos. Foveation was applied by increasing compression with increased eccentricity. In the 2D study, each video was of resolution $7680\times 3840$ and was viewed and quality-rated by 36 subjects, while in the 3D study, each video was of resolution $5376\times 5376$ and rated by 34 subjects. Both studies were conducted on top of a foveated video player having low motion-to-photon latency (~50ms). We evaluated different objective image and video quality assessment algorithms, including both FIQA / FVQA algorithms and non-foveated algorithms, on our so called LIVE-Facebook Technologies Foveation-Compressed Virtual Reality (LIVE-FBT-FCVR) databases. We also present a statistical evaluation of the relative performances of these algorithms. The LIVE-FBT-FCVR databases have been made publicly available and can be accessed at https://live.ece.utexas.edu/research/LIVEFBTFCVR/index.html. [ABSTRACT FROM AUTHOR]

With the rapidly increasing resolutions of 360° cameras, head-mounted displays, and live-streaming services, streaming high-resolution panoramic videos over limited-bandwidth networks is becoming a critical challenge. Foveated video streaming can address this rising challenge in the context of eye-tracking-equipped virtual reality head-mounted displays. However, conventional log-polar foveated rendering suffers from a number of visual artifacts such as aliasing and flickering. In this paper, we introduce a new log-rectilinear transformation that incorporates summed-area table filtering and off-the-shelf video codecs to enable foveated streaming of 360° videos suitable for VR headsets with built-in eye-tracking. To validate our approach, we build a client-server system prototype for streaming 360° videos which leverages parallel algorithms over real-time video transcoding. We conduct quantitative experiments on an existing 360° video dataset and observe that the log-rectilinear transformation paired with summed-area table filtering heavily reduces flickering compared to log-polar subsampling while also yielding an additional 10% reduction in bandwidth usage. [ABSTRACT FROM AUTHOR]

Just noticeable difference (JND) models reveal visibility limitation of human visual system (HVS) and exploit perceptual redundancy in the visual signal. Conventional just noticeable difference (JND) model supposes the visual acuity is homogeneous with or without considerations on the isoeccentric locations. Recent research on vision science indicates that the effect of eccentricity on visual sensitivity is not homogeneous across the visual field (VF). Relevant studies have found visual acuity at the isoeccentric locations is better along the horizontal meridian than along the vertical meridian, i.e., horizontal-vertical anisotropy (HVA). Along the vertical meridian, better performance has been reported for the inferior meridian compared to the superior meridian, i.e., vertical-meridian asymmetry (VMA). These properties of HVS should be taken into account to yield a better JND estimation, for better exploiting the perceptual redundancy. In this paper, we use two hemi-ellipses to represent the iso-acuity contour, which can better express the HVA and VMA effects. Moreover, we design psychophysical experiments under different test conditions, where the results show the hemi-ellipse model performs better when compared to the circular model. Finally, we build an asymmetric foveated JND model using two hemi-ellipses for approximation. When comparing with traditional JND models, experimental results demonstrate the effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

Analyzing the perceptual image quality rendered on a head‐mounted display (HMD) is of primary importance to provide a flawless visual quality to VR headset users. In this paper, we propose a new subjective testing method to evaluate the quality of foveated images in HMDs. The images we use are coded by progressively increasing the compression ratio from the fixation point to the periphery. Our method is based on a standardized flickering paradigm which temporally interleaves the original and reconstructed images to make the differences more prominent. Results indicate that observers can identify more visual impairments near the fixation point than in periphery. This new subjective testing approach is, therefore, effective in assessing barely visible image artifacts of foveated imaging in HMDs. [ABSTRACT FROM AUTHOR]

Cybersickness is one of the major roadblocks in the widespread adoption of mixed reality devices. Prolonged exposure to these devices, especially virtual reality devices, can cause users to feel discomfort and nausea, spoiling the immersive experience. Incorporating spatial blur in stereoscopic 3D stimuli has shown to reduce cybersickness. In this paper, we develop a technique to incorporate spatial blur in VR systems inspired by the human physiological system. The technique makes use of concepts from foveated imaging and depth-of-field. The developed technique can be applied to any eye tracker equipped VR system as a post-processing step to provide an artifact-free scene. We verify the usefulness of the proposed system by conducting a user study on cybersickness evaluation. We used a custom-built rollercoaster VR environment developed in Unity and an HTC Vive Pro Eye headset to interact with the user. A Simulator Sickness Questionnaire was used to measure the induced sickness while gaze and heart rate data were recorded for quantitative analysis. The experimental analysis highlighted the aptness of our foveated depth-of-field effect in reducing cybersickness in virtual environments by reducing the sickness scores by approximately 66%.

Infantile nystagmus (IN) describes a regular, repetitive movement of the eyes. A characteristic feature of each cycle of the IN eye movement waveform is a period in which the eyes are moving at minimal velocity. This so-called "foveation" period has long been considered the basis for the best vision in individuals with IN. In recent years, the technology for measuring eye movements has improved considerably, but there remains the challenge of calibrating the direction of gaze in tracking systems when the eyes are continuously moving. Identifying portions of the nystagmus waveform suitable for calibration typically involves time-consuming manual selection of the foveation periods from the eye trace. Without an accurate calibration, the exact parameters of the waveform cannot be determined. In this study, we present an automated method for segmenting IN waveforms with the purpose of determining the foveation positions to be used for calibration of an eye tracker. On average, the "point of regard" was found to be within 0.21° of that determined by hand-marking by an expert observer. This method enables rapid clinical quantification of waveforms and the possibility of gaze-contingent research paradigms being performed with this patient group. [ABSTRACT FROM AUTHOR]

Energy and storage restrictions are relevant variables that software applications should be concerned about when running in low-power environments. In particular, computer vision (CV) applications exemplify well that concern, since conventional uniform image sensors typically capture large amounts of data to be further handled by the appropriate CV algorithms. Moreover, much of the acquired data are often redundant and outside of the application’s interest, which leads to unnecessary processing and energy spending. In the literature, techniques for sensing and re-sampling images in non-uniform fashions have emerged to cope with these problems. In this study, we propose Application-Oriented Retinal Image Models that define a space-variant configuration of uniform images and contemplate requirements of energy consumption and storage footprints for CV applications. We hypothesize that our models might decrease energy consumption in CV tasks. Moreover, we show how to create the models and validate their use in a face detection/recognition application, evidencing the compromise between storage, energy, and accuracy.

Pan-tilt-zoom (PTZ) and omnidirectional cameras serve as a video-mediated communication interface for telemedicine. Most cases use either PTZ or omnidirectional cameras exclusively; even when used together, images from the two are shown separately on 2D displays. Conventional foveated imaging techniques may offer a solution for exploiting the benefits of both cameras, i.e., the high resolution of the PTZ camera and the wide field-of-view of the omnidirectional camera, but displaying the unified image on a 2D display would reduce the benefit of “omni-” directionality. In this paper, we introduce a foveated imaging pipeline designed to support virtual reality head-mounted displays (HMDs). The pipeline consists of two parallel processes: one for estimating parameters for the integration of the two images and another for rendering images in real time. A control mechanism for placing the foveal region (i.e., high-resolution area) in the scene and zooming is also proposed. Our evaluations showed that the proposed pipeline achieved, on average, 17 frames per second when rendering the foveated view on an HMD, and showed angular resolution improvement on the foveal region compared with the omnidirectional camera view. However, the improvement was less significant when the zoom level was 8× and more. We discuss possible improvement points and future research directions.

Highlights • Just noticeable difference (JND) increases when viewing depth increase in the VR environment. • Considering depth effect on visibility threshold can add more unperceived noises in images. • Foveation effect that JND increases with eccentricity is also observed in the VR environment. Abstract In this paper, we develop a joint foveation-depth just-noticeable-difference (FD-JND) model to quantify the perceptual redundancy of image in the VR display environment. The proposed FD-JND model is developed with considerations on the effects of both foveation and depth. More specifically, experiments for the VR environment on synthesized stimuli are conducted based on luminance masking and contrast masking and the FD-JND model is developed accordingly. Subjective quality discrimination experiments between the noise contaminated images and original ones validate favorableness of the proposed FD-JND model. [ABSTRACT FROM AUTHOR]

In image compression and transmission, two basic data redundancies can be identified and exploited: statistical redundancy and perceptual redundancy. Statistical redundancy has been studied for a very long time and has been exploited in most image coding schemes. However, perceptual redundancy has not yet been completely exploited. Furthermore, perceptual redundancy is difficult to exploit by using digital coding techniques because power allocation is performed at the bit level for digital coding. However, as one of one-size-fits-all technique, soft transmission is suitable to exploit the perceptual redundancy because its power allocation is directly applied at the pixel level instead of the bit level. In this paper, we propose a novel image transmission scheme, named FoveaCast , for single-input single-output or multi-input multi-output broadcast systems that effectively utilizes the foveation characteristic of human vision and analog coding techniques to achieve higher visual perceptual quality. Hence, our scheme possesses not only high visual perceptual quality but also graceful quality adaptation. Experimental evaluations show that compared with three existing digital schemes and two existing analog schemes, SoftCast and ParCast, FoveaCast achieves better visual perceptual performance. Meanwhile, it achieves a graceful quality variation along with channel variation, behaving just like SoftCast and ParCast. [ABSTRACT FROM AUTHOR]

The caudal fastigial nuclei (cFN) are the output nuclei by which the medio-posterior cerebellum influences the production of visual saccades. We investigated in two head-restrained monkeys their contribution to the generation of interceptive saccades toward a target moving centrifugally by analyzing the consequences of a unilateral inactivation (10 injection sessions). We describe here the effects on saccades made toward a centrifugal target that moved along the horizontal meridian with a constant (10, 20, or 40°/s), increasing (from 0 to 40°/s over 600 ms), or decreasing (from 40 to 0°/s over 600 ms) speed. After muscimol injection, the monkeys were unable to foveate the current location of the moving target. The horizontal amplitude of interceptive saccades was reduced during contralesional target motions and hypermetric during ipsilesional ones. For both contralesional and ipsilesional saccades, the magnitude of dysmetria increased with target speed. However, the use of accelerating and decelerating targets revealed that the dependence of dysmetria upon target velocity was not due to the current velocity but to the required amplitude of saccade. We discuss these results in the framework of two hypotheses, the so-called "dual drive" and "bilateral" hypotheses. [ABSTRACT FROM AUTHOR]

Humans visually process the world with varying spatial resolution and can program their eye movements optimally to maximize information acquisition for a variety of everyday tasks. Diseases such as macular degeneration can change visual sensory processing, introducing central vision loss (a scotoma). However, humans can learn to direct a new preferred retinal location to regions of interest for simple visual tasks. Whether such learned compensatory saccades are optimal and generalize to more complex tasks, which require integrating information across a large area of the visual field, is not well understood. Here, we explore the possible effects of central vision loss on the optimal saccades during a face identification task, using a gaze-contingent simulated scotoma. We show that a new foveated ideal observer with a central scotoma correctly predicts that the human optimal point of fixation to identify faces shifts from just below the eyes to one that is at the tip of the nose and another at the top of the forehead. However, even after 5000 trials, humans of both sexes surprisingly do not change their initial fixations to adapt to the new optimal fixation points to faces. In contrast, saccades do change for tasks such as object following and to a lesser extent during search. Our findings argue against a central brain motor-compensatory mechanism that generalizes across tasks. They instead suggest task specificity in the learning of oculomotor plans in response to changes in front-end sensory processing and the possibility of separate domain-specific representations of learned oculomotor plans in the brain. [ABSTRACT FROM AUTHOR]

This paper presents a real-time foveation technique proposed as an alternative method for image obfuscation while simultaneously preserving privacy in face deidentification. Relevance of the proposed technique is discussed through a comparative study of the most common distortions methods in face images and an assessment on performance and effectiveness of privacy protection. All the different techniques presented here are evaluated when they go through a face recognition software. Evaluating the data utility preservation was carried out under gender and facial expression classification. Results on quantifying the tradeoff between privacy protection and image information preservation at different obfuscation levels are presented. Comparative results using the facial expression subset of the FERET database show that the technique achieves a good tradeoff between privacy and awareness with 30% of recognition rate and a classification accuracy as high as 88% obtained from the common figures of merit using the privacy-awareness map. [ABSTRACT FROM AUTHOR]

Background: An emerging trend in visual information processing is toward incorporating some interesting properties of the ventral stream in order to account for some limitations of machine learning algorithms. Selective attention and cortical magnification are two such important phenomena that have been the subject of a large body of research in recent years. In this paper, we focus on designing a new model for visual acquisition that takes these important properties into account. Methods: We propose a new framework for visual information acquisition and representation that emulates the architecture of the primate visual system by integrating features such as retinal sampling and cortical magnification while avoiding spatial deformations and other side effects produced by models that tried to implement these two features. It also explicitly integrates the notion of visual angle, which is rarely taken into account by vision models. We argue that this framework can provide the infrastructure for implementing vision tasks such as object recognition and computational visual attention algorithms. Results: To demonstrate the utility of the proposed vision framework, we propose an algorithm for bottom-up saliency prediction implemented using the proposed architecture. We evaluate the performance of the proposed model on the MIT saliency benchmark and show that it attains state-of-the-art performance, while providing some advantages over other models. Conclusion: Here is a summary of the main contributions of this paper: (1) Introducing a new bio-inspired framework for visual information acquisition and representation that offers the following properties: (a) Providing a method for taking the distance between an image and the viewer into account. This is done by incorporating a visual angle parameter which is ignored by most visual acquisition models. (b) Reducing the amount of visual information acquired by introducing a new scheme for emulating retinal sampling and cortical magnification effects observed in the ventral stream. (2) Providing a concrete application of the proposed framework by using it as a substrate for building a new saliency-based visual attention model, which is shown to attain state-of-the-art performance on the MIT saliency benchmark. (3) Providing an online Git repository that implements the introduced framework that is meant to be developed as a scalable, collaborative project. [ABSTRACT FROM AUTHOR]

Foveated sight as observed in some raptor eyes is a motivation for artificial imaging systems requiring both wide fields of view as well as specific embedded regions of higher resolution. These foveated optical imaging systems are applicable to many acquisition and tracking tasks and as such are often required to be relatively portable and operate in real-time. Two approaches to achieve foveation have been explored in the past: optical system design and back-end data processing. In this paper, these previous works are compiled and used to build a framework for analyzing and designing practical foveated imaging systems. While each approach (physical control of optical distortion within the lens design process, and post-processing image re-sampling) has its own pros and cons, it is concluded that a combination of both techniques will further spur the development of more versatile, flexible, and adaptable foveated imaging systems in the future.

This work explores ways to bypass the fundamental image quality limitations of displays using Computer Generated Holograms (CGHs) and, specifically, the high-frequency noise associated with phase-only holograms. Although there is a wealth of literature on building experimental holographic systems, there are no user studies to assess the performance of a holographic system projecting a dynamic two-dimensional image. In this study, 18 participants blindly compared three groups of images displayed on a conventional monitor. The first group contained the original image, the second the simulated holographic reconstruction of the original image, and the third group had the foveated reconstruction of the original image, based on the pupil position. Holograms in the second group were computed using the Fienup algorithm and the third group using the Fienup with Perceptual Don't Care Areas (FiPDoC) algorithm, a novel algorithm that uses eye tracking to optimize image quality in CGHs. The aim of the study was to find out if the holographic display, assuming an ideal hardware, can be as good as a conventional display and whether eye tracking can help this goal. Most participants distinguished between the original image and the un-foveated simulated reconstruction. However, the participants could not differentiate between the original image and the foveated reconstruction. Thus, foveation may be essential in designing and building the first commercial holographic displays. • Holographically projected images cannot be noise free due to theoretical limitations. • Holographic images often "bury" noise in areas of high-frequency content. • Foveation can reduce perceived noise in images with uniform intensity or slow-changing features. • Holographic head-mounted displays can utilize on-board eye-tracking for foveation. • In a user study, most reconstructions were indistinguishable from the original images [ABSTRACT FROM AUTHOR]

Combining the principles behind Compressed Sensing theory with a wavelet-based implementation of a foveation operator inspired by the human visual system yields significant compression performances on both 1D and 2D signals. The solution provides spatially variable quality of the reconstructed information, enabling better approximation on specific regions of interest. Four distinct algorithms are compared in terms of reconstruction error and compression ratio on a set of ECG records and natural images. [ABSTRACT FROM AUTHOR]

Vision in foveate animals is an active process that requires rapid and constant decision-making. For example, when a new object appears in the visual field, we can quickly decide to inspect it by directing our eyes to the object's location. We studied the contribution of primate area V4 to these types of rapid foveation decisions. Animals performed a reaction time task that required them to report when any shape appeared within a peripherally-located noisy stimulus by making a saccade to the stimulus location. We found that about half of the randomly sampled V4 neurons not only rapidly and precisely represented the appearance of this shape, but they were also predictive of the animal's saccades. A neuron's ability to predict the animal's saccades was not related to the specificity with which the cell represented a single type of shape but rather to its ability to signal whether any shape was present. This relationship between sensory sensitivity and behavioral predictiveness was not due to global effects such as alertness, as it was equally likely to be observed for cells with increases and decreases in firing rate. Careful analysis of the timescales of reliability in these neurons implies that they reflect both feed forward and feedback shape detecting processes. In approximately 7% of our recorded sample, individual neurons were able to predict both the delay and precision of the animal's shape detection performance. This suggests that a subset of V4 neurons may have been directly and causally contributing to task performance and that area V4 likely plays a critical role in guiding rapid, form-based foveation decisions. [ABSTRACT FROM AUTHOR]

To achieve clear binocular vision, neural processes that accomplish accommodation and vergence are performed via two collaborative, cross-coupled processes: accommodation-vergence (AV) and vergence-accommodation (VA). However, when people watch stereo images on stereoscopic displays, normal neural functioning may be disturbed owing to anomalies of the cross-link gains. These anomalies are likely the main cause of visual discomfort experienced when viewing stereo images, and are called Accommodation-Vergence Mismatches (AVM). Moreover, the absence of any useful accommodation depth cues when viewing 3D content on a flat panel (planar) display induces anomalous demands on binocular fusion, resulting in possible additional visual discomfort. Most prior efforts in this direction have focused on predicting anomalies in the AV cross-link using measurements on a computed disparity map. We further these contributions by developing a model that accounts for both accommodation and vergence, resulting in a new visual discomfort prediction algorithm dubbed the 3D-AVM Predictor. The 3D-AVM model and algorithm make use of a new concept we call local 3D bandwidth (BW) which is defined in terms of the physiological optics of binocular vision and foveation. The 3D-AVM Predictor accounts for anomalous motor responses of both accommodation and vergence, yielding predictive power that is statistically superior to prior models that rely on a computed disparity distribution only. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a new limit that promises theoretically achievable data reduction ratios up to approximately 9:1 with no perceptual loss in typical scenarios. Also, we introduce a novel Gaussian foveation scheme that provides experimentally achievable gains up to approximately 2 times the compression ratio of typical compression schemes with less perceptual loss than in typical transmissions. Both the proposed limit and foveation scheme shares the same background material: a model of image projection onto the retina; a model of cones distribution; and, subsequently, a proposed pointwise retina-based constraint called pixel efficiency. Quantitatively, the lattermost is globally processed to reveal the perceptual efficiency of a display. Analytical results indicate that in general the perceptual efficiency of displays are low for typical image sizes and viewing distances. Qualitatively, the pixel efficiency is used together with a lossy parameter to locally control the spatial resolution of a foveated image. Practical results show that proper use of the lossy parameter in the foveation filtering can increase the subjective quality of images. [ABSTRACT FROM PUBLISHER]

The concentration of the cones and ganglion cells is much higher in the fovea than the rest of the retina. This non-uniform sampling results in a retinal image that is sharp at the fixation point, where a person is looking, and blurred away from it. This difference between the sampling rates at the different spatial locations presents us with the question of whether we can employ this biological characteristic to achieve better image compression. This can be achieved by compressing an image less at the fixation point and more away from it. It is, however, known that the vision system employs more that one fixation to look at a single scene which presents us with the problem of combining images pertaining to the same scene but exhibiting different spatial contrasts. This article presents an algorithm to combine such a series of images by using image fusion in the gradient domain. The advantage of the algorithm is that unlike other algorithms that compress the image in the spatial domain our algorithm results in no artifacts. The algorithm is based on two steps, in the first we modify the gradients of an image based on a limited number of fixations and in the second we integrate the modified gradient. Results based on measured and predicted fixations verify our approach. [ABSTRACT FROM AUTHOR]

Abstract: Visual attention is a very important task in autonomous robotics, but, because of its complexity, the processing time required is significant. We propose an architecture for feature selection using foveated images that is guided by visual attention tasks and that reduces the processing time required to perform these tasks. Our system can be applied in bottom–up or top–down visual attention. The foveated model determines which scales are to be used on the feature extraction algorithm. The system is able to discard features that are not extremely necessary for the tasks, thus, reducing the processing time. If the fovea is correctly placed, then it is possible to reduce the processing time without compromising the quality of the tasks' outputs. The distance of the fovea from the object is also analyzed. If the visual system loses the tracking in top–down attention, basic strategies of fovea placement can be applied. Experiments have shown that it is possible to reduce up to 60% the processing time with this approach. To validate the method, we tested it with the feature algorithm known as speeded up robust features (SURF), one of the most efficient approaches for feature extraction. With the proposed architecture, we can accomplish real time requirements of robotics vision, mainly to be applied in autonomous robotics. [Copyright &y& Elsevier]

Background: Current methods of determining licence retainment or cancellation is through on-road driving tests. Previous research has shown that occupational therapists frequently assess drivers' visual attention while sitting in the back seat on the opposite side of the driver. Since the eyes of the driver are not always visible, assessment by eye contact becomes problematic. Such procedural drawbacks may challenge validity and reliability of the visual attention assessments. In terms of correctly classified attention, the aim of the study was to establish the accuracy and the inter-rater reliability of driving assessments of visual attention from the back seat. Furthermore, by establishing eye contact between the assessor and the driver through an additional mirror on the wind screen, the present study aimed to establish how much such an intervention would enhance the accuracy of the visual attention assessment. Methods: Two drivers with Parkinson's disease (PD) and six control drivers drove a fixed route in a driving simulator while wearing a head mounted eye tracker. The eye tracker data showed where the foveal visual attention actually was directed. These data were time stamped and compared with the simultaneous manual scoring of the visual attention of the drivers. In four of the drivers, one with Parkinson's disease, a mirror on the windscreen was set up to arrange for eye contact between the driver and the assessor. Interrater reliability was performed with one of the Parkinson drivers driving, but without the mirror. Results: Without mirror, the overall accuracy was 56% when assessing the three control drivers and with mirror 83%. However, for the PD driver without mirror the accuracy was 94%, whereas for the PD driver with a mirror the accuracy was 90%. With respect to the inter-rater reliability, a 73% agreement was found. Conclusion: If the final outcome of a driving assessment is dependent on the subcategory of a protocol assessing visual attention, we suggest the use of an additional mirror to establish eye contact between the assessor and the driver. The clinicians' observations on-road should not be a standalone assessment in driving assessments. Instead, eye trackers should be employed for further analyses and correlation in cases where there is doubt about a driver's attention. [ABSTRACT FROM AUTHOR]

We studied the mechanisms of oscillopsia suppression in subjects with infantile nystagmus syndrome, fusion maldevelopment nystagmus syndrome, and acquired nystagmus (AN). Hypothetical possibilities for perceptual stability were the following: (1) epochs of clear and stable vision during foveation periods of nystagmus waveforms; (2) cancellation by efference copy of motor output; (3) a combination of the effects of both foveation-period stability and efference-copy cancellation; or (4) elevated motion-detection threshold and vision suppression. Observations, studies, and models of oscillopsia suppression allowed comparison of these possibilities. Data from individual subjects supported some of the putative hypotheses. However, only one hypothesis remained viable that could explain how all subjects maintained perceptual stability despite their different nystagmus types, waveforms, and variability. Robust suppression of oscillopsia was only possible using efference-copy feedback of the motor output containing these specific nystagmus signals to cancel that motion from the retinal error signals. In cases of AN, where oscillopsia could not be suppressed, the deficit was postulated to interfere with or lie outside of this efference-copy feedback loop. [ABSTRACT FROM AUTHOR]

Modeling the human visual system (HVS) is a challenging task but is necessary since it plays a crucial role in various image and video processing applications. In this paper, we construct a more complete Just Noticeable Distortion (JND) model for videos, where the foveation and the motion suppression factors are accounted for, in addition to the luminance adaptation, the contrast masking and the temporal masking effects which are considered in some existing JND models. JND is the maximum possible error in the signal which is not perceptible to the human eyes. Foveation is another significant feature of the HVS. It is based on the fact that the perceived scene is sampled spatially in a nonuniform manner by the HVS. The point of fixation corresponding to the observer's gaze in the image has the highest visual sensitivity and the sensitivity decreases as we move away from it. One of the key properties of the human eye, the motion suppression effect, exploits the role of contextual motion in a video. It is determined by the absolute and relative velocities of different regions in a video. Experimental results showed that the JND estimated by our model is consistent with the HVS. An accurate estimate of the visibility threshold can be used to increase the performance of many video and image processing applications like compression, objective perceptual quality metrics, watermarking and visual data enhancement and restoration. [ABSTRACT FROM AUTHOR]

This paper reports a 176\times144-pixel smart image sensor designed and fabricated in a 0.35\,\mu\ m CMOS-OPTO process. The chip implements a massively parallel focal-plane processing array which can output different simplified representations of the scene at very low power. The array is composed of pixel-level processing elements which carry out analog image processing concurrently with photosensing. These processing elements can be grouped into fully-programmable rectangular-shape areas by loading the appropriate interconnection patterns into the registers at the edge of the array. The targeted processing can be thus performed block-wise. Readout is done pixel-by-pixel in a random access fashion. On-chip 8b ADC is provided. The image processing primitives implemented by the chip, experimentally tested and fully functional, are scale space and Gaussian pyramid generation, fully-programmable multiresolution scene representation—including foveation—and block-wise energy-based scene representation. The power consumption associated to the capture, processing and A/D conversion of an image flow at 30 fps, with full-frame processing but reduced frame size output, ranges from 2.7 mW to 5.6 mW, depending on the operation to be performed. [ABSTRACT FROM PUBLISHER]

Abstract: This paper considers the problem of video streaming in low bandwidth networks and presents a complete framework that is inspired by the fovea–periphery distinction of biological vision systems. First, an application specific attention function that serves to find the important small regions in a given frame is constructed a priori using a back-propagation neural network that is optimized combinatorially. Given a specific application, the respective attention function partitions each frame into foveal and periphery regions and then a spatial–temporal pre-processing algorithm encodes the foveal regions with high spatial resolution while the periphery regions are encoded with lower spatial and temporal resolution. Finally, the pre-processed video sequence is streamed using a standard streaming server. As an application, we consider the transmission of human face videos. Our experimental results indicate that even with limited amount of training, the constructed attention function is able to determine the foveal regions which have improved transmission quality while the peripheral regions have an acceptable degradation. [ABSTRACT FROM AUTHOR]

Abstract: We updated and extended the functionality of the eXpanded Nystagmus Acuity Function (NAFX), for application under more diverse circumstances, improving its clinical predictive value. The original NAFX “τ-surface” of minimum-necessary-foveation times had been individually calculated for each combination of position and velocity limits. We have replaced it with an idealized mathematical function that repairs the irregularities in its surface due to idiosyncrasies in the subject data used for the initial calculations. To extend applicability to multiplanar data, we combine horizontal and vertical eye-movement data into a single waveform using vector summation. Torsional eye movements have little effect on visual acuity and are ignored. Age-related visual acuity relationships, derived from population data, more accurately relate the NAFX value to acuity for individual patients. Using the same patient fixation data that established the original NAF and NAFX functions, we verified that the updated NAFX yielded equivalent results for uniplanar data. For biplanar data, the results were also comparable to those of uniplanar data of the same magnitude. The updated NAFX yields greater accuracy in prediction of potential visual acuity for subjects of all ages, for uniplanar and multiplanar nystagmus, extending the objective, direct measure of post-therapy waveform improvement, allowing selection of the best therapy for a wider range of nystagmus patients. [Copyright &y& Elsevier]

Biological vision systems have inspired and will continue to inspire the development of computer vision systems. One biological tendency that has never been exploited is the symbiotic relationship between foveation and uncalibrated active, binocular vision systems. The primary goal of any binocular vision system is the correspondence of the two retinal images. For calibrated binocular rigs the search for corresponding points can be restricted to epipolar lines. In an uncalibrated system the precise geometry is unknown. However, the set of possible geometries can be restricted to some reasonable range; and consequently, the search for matching points can be confined to regions delineated by the union of all possible epipolar lines over all possible geometries. We call these regions epipolar spaces. The accuracy and complexity of any correspondence algorithm is directly proportional to the size of these epipolar spaces. Consequently, the introduction of a spatially variant foveation strategy that reduces the average area per epipolar space is highly desirable. This paper provides a set of sampling theorems that offer a path for designing foveation strategies that are optimal with respect to average epipolar area. [ABSTRACT FROM AUTHOR]

Abstract: We wished to determine the influential factors for Infantile Nystagmus Syndrome (INS) subjects’ ability to acquire and pursue moving targets using predictions from the behavioral Ocular Motor System (OMS) model and data from INS subjects. Ocular motor simulations using a behavioral OMS model were performed in MATLAB Simulink. Eye-movement recordings were performed using a high-speed digital video system. We studied five INS subjects who pursued a 10°/s ramp target to both left and right. We measured their target-acquisition times based on position criteria. The following parameters were studied: Lt (measured from the target-ramp initiation to the first on-target foveation period), target pursuit direction, and foveation-period pursuit gain. Analyses and simulations were performed in MATLAB environment using OMLAB software (OMtools, download from http://www.omlab.org). Ramp-target timing influenced target-acquisition time; the closer to the intrinsic saccades in the waveform the ramp stimuli started, the longer was Lt. However, arriving at the target position may not guarantee its foveation. Foveation-period pursuit gains vs. target or slow-phase direction had an idiosyncratic relationship for each subject. Adjustments to the model’s Fixation subsystem reproduced the idiosyncratic foveation-period pursuit gains; the gain of the Smooth Pursuit subsystem was maintained at its normal value. The model output predicted a steady-state error when target initiation occurred during intrinsic saccades, consistent with human data. We conclude that INS subjects acquire ramp targets with longer latency for target initiations during or near the intrinsic saccades, consistent with the findings in our step-stimuli timing study. This effect might be due to the interaction between the saccadic and pursuit systems. The combined effects of target timing and Fixation-subsystem gain determined how fast and how well the INS subjects pursued ramp stimuli during their foveations periods (i.e., their foveation-period pursuit gain). The OMS model again demonstrated its behavioral characteristics and prediction capabilities (e.g., steady-state error) and revealed an important interaction between the Fixation and Smooth Pursuit subsystems. [Copyright &y& Elsevier]

Abstract: Object recognition is one of the most important tasks of the visual cortex. Even though it has been closely studied in the field of computer vision and neuroscience, the underlying processes in the visual cortex are not completely understood. A model that lately has gained attention is the HMAX model, which describes a feedforward hierarchical structure. This model shows a degree of scale and translation invariance. Our work explores and compares the HMAX model with a simpler model for object recognition emulating simple cells in the primary visual cortex, V1. This model shows a better performance than the HMAX model for translation and scale invariance experiments when an attentional mechanism is employed in realistic conditions. [Copyright &y& Elsevier]

The ability to automatically detect visually interesting regions in images has many practical applications, especially in the design of active machine vision and automatic visual surveillance systems. Analysis of the statistics of image features at observers' gaze can provide insights into the mechanisms of fixation selection in humans. Using a foveated analysis framework, we studied the statistics of four low-level local image features: luminance, contrast, and bandpass outputs of both luminance and contrast, and discovered that image patches around human fixations had, on average, higher values of each of these features than image patches selected at random. Contrast-bandpass showed the greatest difference between human and random fixations, followed by luminance-bandpass, RMS contrast, and luminance. Using these measurements, we present a new algorithm that selects image regions as likely candidates for fixation. These regions are shown to correlate well with fixations recorded from human observers. [ABSTRACT FROM AUTHOR]

This study investigated perceptual disruptions in gaze-contingent multiresolutional displays (GCMRDs) due to delays in updating the center of highest resolution after an eye movement. GCMRDs can be used to save processing resources and transmission bandwidth in many types of single-user display applications, such as virtual reality, video-telephony, simulators, and remote piloting. The current study found that image update delays as late as 60 ms after an eye movement did not significantly increase the detectability of image blur and/or motion transients due to the update. This is good news for designers of GCMRDs, since 60 ms is ample time to update many GCMRDs after an eye movement without disrupting perception. The study also found that longer eye movements led to greater blur and/or transient detection due to moving the eyes further into the low-resolution periphery, effectively reducing the image resolution at fixation prior to the update. In GCMRD applications where longer saccades are more likely (e.g., displays with relatively large distances between objects), this problem could be overcome by increasing the size of the region of highest resolution. [ABSTRACT FROM AUTHOR]

Advancements in graphics hardware have allowed development of hardware-accelerated imaging displays. This article reviews techniques for real-time simulation of arbitrary visual fields over still images and video. The goal is to provide the vision sciences and perceptual graphics communities techniques for the investigation of fundamental processes of visual perception. Classic gaze-contingent displays used for these purposes are reviewed and for the first time a pixel shader is introduced for display of a high-resolution window over peripherally degraded stimulus. The pixel shader advances current state-of-the-art by allowing real-time processing of still or streamed images, obviating the need for preprocessing or storage. [ABSTRACT FROM AUTHOR]

Abstract: Analysis of the statistics of image features at observers’ gaze can provide insights into the mechanisms of fixation selection in humans. Using a foveated analysis framework, in which image patches were analyzed at the resolution corresponding to their eccentricity from the prior fixation, we studied the statistics of four low-level local image features: luminance, RMS contrast, and bandpass outputs of both luminance and contrast, and discovered that the image patches around human fixations had, on average, higher values of each of these features at all eccentricities than the image patches selected at random. Bandpass contrast showed the greatest difference between human and random fixations, followed by bandpass luminance, RMS contrast, and luminance. An eccentricity-based analysis showed that shorter saccades were more likely to land on patches with higher values of these features. Compared to a full-resolution analysis, foveation produced an increased difference between human and random patch ensembles for contrast and its higher-order statistics. [Copyright &y& Elsevier]

Wavelet image coding exhibits a robust error resilience performance by utilizing a naturally layered bitstream construction over a band-limited channel. In this letter, a new measure that appears to provide a better assessment of visual entropy for comparing and evaluating progressive image coders is defined based on a visual weight over the wavelet domain. This visual weight is characterized by the human visual system (HVS) over the frequency and spatial domains and is then utilized as a criterion for determining the coding order of wavelet coefficients, resulting in improved visual quality. A transmission gain, which is expressed by visual entropy, of up to about 23% can be obtained at a normalized channel throughput of about 0.3. In accordance with the subjective visual quality, a relatively high gain can be obtained at a low channel capacity. [ABSTRACT FROM AUTHOR]

In preverbal children, it is difficult to assess visual acuity accurately. If nystagmus waveforms can be used to predict visual acuity in these young patients, then they would be a useful additional clinical tool. Existing acuity predictors were assessed in 10 adults with idiopathic early onset nystagmus. The most reliable predictor in this group of patients was mean velocity of the waveform. The mean velocity of the waveform may be used in preverbal children to give an estimate of best potential visual acuity in adulthood, and estimated best potential visual acuity at the current age. [ABSTRACT FROM AUTHOR]

Abstract: Video transcoding is a key technology to support video communications over heterogeneous networks. Although quite a bit of research effort has been made in video transcoding due to its wide applications, most video transcoding techniques proposed in the literature are optimized based on the simple mean squared error (MSE) metric which does not correlate well with the human visual perception. In this paper, foveation, a property of the HVS, is exploited in video transcoding. The proposed foveation embedded DCT domain video transcoding can reduce the bit rate without compromising visual quality or achieve better subjective quality for a given bit rate by shaping the compression distortion according to the foveated contrast sensitivity function of the HVS. In addition, fast algorithms for video foveation filtering and DCT domain inverse motion compensation are developed, which significantly improve the efficiency of video transcoding. [Copyright &y& Elsevier]

We have developed novel progressive scalable region-of-interest (ROI) image compression schemes with rate-distortion-complexity tradeoff based on vector quantization. Residual vector quantization (RVQ) equips the encoder with a multi-resolution apparatus which is useful for rate-distortion tradeoff. Having all advantages of RVQ, jointly suboptimized RVQ provides a distortion-complexity adjustment. The systems are unbalanced in the sense that the decoder has less computational requirements than the encoder. The proposed jointly suboptimized RVQ method provides an interactive tool for fast ROI-based browsing from image archives. [ABSTRACT FROM PUBLISHER]

Abstract: Commonly, when an individual with congenital nystagmus (CN) performs a visually demanding task their nystagmus intensifies and their visual acuity decreases, probably due to poorer foveation. However, the relationship between fixation attempt and nystagmus waveform has never been quantified. In this study 14 CN subjects viewed a Landolt C of varying orientation and size. They indicated its orientation via a button array whilst eye movements were recorded. Foveation was uncorrelated with optotype size. These results suggest that CN is not exacerbated by visual demand per se rather the need to do something visually demanding of importance to the individual. [Copyright &y& Elsevier]

Foveated video streams are created by selectively varying the spatio-temporal resolution of video data according to the assumed or measured fixation patterns of the intended viewers. The significant potential of foveated video lies in the considerable entropy reduction relative to the original video data while minimizing the loss of apparent visual information. By exploiting new human-machine interface techniques, such as visual eyetrackers, combined with foveated video compression and communication protocols, more efficient visual multimedia services operating over low bandwidths should become available in the near future. In this paper, we introduce a prototype foveated visual communication system suitable for implementation as a core element of an interactive multimedia wireless communication environment. We demonstrate the benefit of using foveation in noisy wireless low bandwidth applications, using measured fading statistics from the downtown area of Austin, Texas as an example. Based on a maximum source throughput criterion, the source and channel video transmission rates and the target video coding bit rate are adaptively decided according to the channel dynamics. In the simulations, we use the channel throughput, the spatial/temporal resolution, and the transmission delay as criteria to compare the performance of the foveated approach relative to normal (non-foveated) video. The results clearly underline the significant potential of foveated video communication protocols for wireless multimedia applications. [ABSTRACT FROM AUTHOR]