Search

This article describes a dataset of synthetic images representing biological scenery as captured by a Fourier Lightfield Microscope (FLMic). It includes 22,416 images related to eight scenes composed of 3D models of objects typical for biological samples, such as red blood cells and bacteria, and categorized into Cells and Filaments groups. For each scene, two types of image data structures are provided: 51 × 51 Elemental Images (EIs) representing Densely Sampled Light Fields (DSLF) and 201 images composing Z-Scans of the scenes. Auxiliary data also includes information about camera intrinsic and extrinsic calibration parameters, object descriptions, and MATLAB scripts for camera pose compensation. The images have been generated using Blender. The dataset can be used to develop and assess methods for volumetric reconstruction from Light Field (LF) images captured by a FLMic.

We address the problem of efficient content-adaptive superpixel segmentation. Instead of adapting the size and/or amount of superpixels to the image content, we propose a warpingtransform that makes the image content more suitable to be segmented into regular superpixels. Regular superpixels in the warped image induce content-adaptive superpixels in the original image with improved segmentation accuracy. To efficiently compute the warping transform, we develop an iterative coarse-to-fine optimization procedure and employ a parallelization strategy allowing for a speedy GPU-based implementation. This solution works as a simple ‘add-on’ framework over an underlying segmentation algorithm and requires no additional parameters. Compared to the state-of-the-art methods, our approach provides competitive quality results and achieves a better time-accuracy trade-off. We further demonstrate the effectiveness of our method with an application to disparity estimation.

Depth of field is an important factor of imaging systems that highly affects the quality of the acquired spatial information. Extended depth of field (EDoF) imaging is a challenging ill-posed problem and has been extensively addressed in the literature. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element (DOE) and we achieve deblurring through a convolutional neural network. Thanks to the end-to-end differentiable modeling of optical image formation and computational post-processing, we jointly optimize the optical design, i.e., DOE, and the deblurring through standard gradient descent methods. Based on the properties of the underlying refractive lens and the desired EDoF range, we provide an analytical expression for the search space of the DOE, which is instrumental in the convergence of the end-to-end network. We achieve superior EDoF imaging performance compared to the state of the art, where we demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging. acceptedVersion

Holography is usually considered as the ultimate way to visually reproduce a three-dimensional scene. Computer-generated holography constitutes an important branch of holography, which enables visualization of artificially generated scenes as well as real three-dimensional scenes recorded under white-light illumination. In this article, we present a comprehensive survey of methods for synthesis of computer-generated holograms, classifying them into two broad categories: wavefront-based methods and ray-based methods. We examine their modern implementations in terms of the quality of reconstruction and computational efficiency. As it is an integral part of computer-generated holography, we devote a special section to speckle suppression, which is also discussed under two categories following the classification of underlying computer-generated hologram methods.

We present a general framework which can handle different processing stages of the three-dimensional (3D) scene representation referred to as 'view-plus-depth' (V+Z). The main component of the framework is the relation between the depth map and the super-pixel segmentation of the color image. We propose a hierarchical super-pixel segmentation which keeps the same boundaries between hierarchical segmentation layers. Such segmentation allows for a corresponding depth segmentation, decimation and reconstruction with varying quality and is instrumental in tasks such as depth compression and 3D data fusion. For the latter we utilize a cross-modality reconstruction filter which is adaptive to the size of the refining super-pixel segments. We propose a novel depth encoding scheme, which includes specific arithmetic encoder and handles misalignment outliers. We demonstrate that our scheme is especially applicable for low bit-rate depth encoding and for fusing color and depth data, where the latter is noisy and with lower spatial resolution. publishedVersion

Fiducial marker-based tracking is an effective method for pose estimation in coffined environments, such as the International Thermonuclear Experimental Reactor. In this paper, we propose a novel framework for marker detection and identification that is moderately robust to occlusion, even while using a relatively small number of marks. The proposed approach (ITER-Tag) consists of a hybrid pipeline that extracts marker candidates from images using classical methods and identifies uncoded markers using a shallow feed forward neural network. The method requires minimal parameter tuning, data collection and annotation. The methods can be easily adapted to different use cases with varying number and positions of the marks. We test the robustness of our approach in three different use cases in ITER’s divertor, using either retro reflective markers or laser engravings and achieve high detectability rates. We demonstrate how the proposed approach can be used to accurately and robustly retrieve the six-degree-of-freedom pose of the targets. acceptedVersion

Hyperspectral imaging critically serves for various fields such as remote sensing, biomedical and agriculture. Its potential can be exploited to a greater extent when combined with deep learning methods, which improve the reconstructed hyperspectral image quality and reduce the processing time. In this paper, we propose a novel snapshot hyperspectral imaging system using optimized diffractive optical element and color filter along with the residual dense network. We evaluate our method through simulations considering the effects of each optical element and noise. Simulation results demonstrate high-quality hyperspectral image reconstruction capabilities through the proposed computational hyperspectral camera. acceptedVersion

The spatio-angular resolution of a light field (LF) display is a crucial factor for delivering adequate spatial image quality and eliciting an accommodation response. Previous studies have modelled retinal image formation with an LF display and evaluated whether accommodation would be evoked correctly. The models were mostly based on ray-tracing and a schematic eye model, which pose computational complexity and inaccurately represent the human eye population's behaviour. We propose an efficient wave-optics-based framework to model the human eye and a general LF display. With the model, we simulated the retinal point spread function (PSF) of a point rendered by an LF display at various depths to characterise the retinal image quality. Additionally, accommodation responses to rendered LF images were estimated by computing the visual Strehl ratio based on the optical transfer function (VSOTF) from the PSFs. We assumed an ideal LF display that had an infinite spatial resolution and was free from optical aberrations in the simulation. We tested images rendered at 0--4 dioptres of depths having angular resolutions of up to 4x4 viewpoints within a pupil. The simulation predicted small and constant accommodation errors, which contradict the findings of previous studies. An evaluation of the optical resolution of the rendered retinal image suggested a trade-off between the maximum resolution achievable and the depth range of a rendered image where in-focus resolution is kept high. The proposed framework can be used to evaluate the upper bound of the optical performance of an LF display for realistically aberrated eyes, which may help to find an optimal spatio-angular resolution required to render a high quality 3D scene., Comment: 24 pages, 12 figures, submitted to Optics Express

We present a computational accommodation-invariant near-eye display, which relies on imaging with coherent light and utilizes static optics together with convolutional neural network-based preprocessing. The network and the display optics are co-optimized to obtain a depth-invariant display point spread function, and thus relieve the conflict between accommodation and ocular vergence cues that typically exists in conventional near-eye displays. We demonstrate through simulations that the computational near-eye display designed based on the proposed approach can deliver sharp images within a depth range of 3 diopters for an effective aperture (eyepiece) size of 10 mm. Thus, it provides a competitive alternative to the existing accommodation-invariant displays. acceptedVersion

We address the problem of 3D scene rendering on projection-based light field displays and optimizing the input display images to obtain the best possible visual output. We discuss a display model comprising a set of projectors, an anisotropic diffuser and a viewing manifold. Based on this model, we render an initial set of projector images to be further optimized for the best perception at a specified set of viewing positions. We propose a least squares method, which minimizes the channel-wise color difference between the generated images for different viewer positions, and their ground-true counterparts. We formulate a constrained optimization problem and solve it iteratively by the descent method. acceptedVersion

We address the problem of 3D scene rendering on near-eye integral imaging displays and evaluation of different rendering methods in terms of human perception. We compare three rendering techniques in terms of perceived spatial resolution at different focused depths, simulating the display in virtual environment and representing the eye through a thin-lens camera model. acceptedVersion

We present a multifocal computational near-eye display that employs a static diffractive optical element (DOE) in tandem with a refractive lens. The DOE is co-optimized with the con-volutional neural network-based preprocessing to achieve desired multifocal display point spread function in an optimal manner. In the simulations, we demonstrate a multifocal display that can deliver sharp images for three distinct depths sampling the dioptric depth range uniformly from 3 diopters to infinity. acceptedVersion

Light field (LF) acquisition devices capture spatial and angular information of a scene. In contrast with traditional cameras, the additional angular information enables novel post-processing applications, such as 3D scene reconstruction, the ability to refocus at different depth planes, and synthetic aperture. In this paper, we present a novel compression scheme for LF data captured using multiple traditional cameras. The input LF views were divided into two groups: key views and decimated views. The key views were compressed using the multi-view extension of high-efficiency video coding (MV-HEVC) scheme, and decimated views were predicted using the shearlet-transform-based prediction (STBP) scheme. Additionally, the residual information of predicted views was also encoded and sent along with the coded stream of key views. The proposed scheme was evaluated over a benchmark multi-camera based LF datasets, demonstrating that incorporating the residual information into the compression scheme increased the overall peak signal to noise ratio (PSNR) by 2 dB. The proposed compression scheme performed significantly better at low bit rates compared to anchor schemes, which have a better level of compression efficiency in high bit-rate scenarios. The sensitivity of the human vision system towards compression artifacts, specifically at low bit rates, favors the proposed compression scheme over anchor schemes. The proposed compression scheme performed significantly better at low bit rates compared to anchor schemes, which have a better level of compression efficiency in high bit-rate scenarios. The sensitivity of the human vision system towards compression artifacts, specifically at low bit rates, favors the proposed compression scheme over anchor schemes. The proposed compression scheme performed significantly better at low bit rates compared to anchor schemes, which have a better level of compression efficiency in high bit-rate scenarios. The sensitivity of the human vision system towards compression artifacts, specifically at low bit rates, favors the proposed compression scheme over anchor schemes.

We present an accommodation-invariant computational neareye display based on the extended depth of field imaging. The eyepiece of the display consists of a diffractive optical element (DOE) that is used in tandem with a conventional refractive lens. The DOE is co-designed with the pre-processing convolutional neural network, which is analogous to the post-processing deblurring networks in image capture. We demonstrate through simulations that such system achieves accommodation-invariant imaging within 2 Diopters depth range without significantly sacrificing spatial resolution. acceptedVersion

The light field and holographic displays constitute two important categories of advanced three-dimensional displays that are aimed at delivering all physiological depth cues of the human visual system, such as stereo cues, motion parallax, and focus cues, with sufficient accuracy. As human observers are the end-users of such displays, the delivered spatial information (e.g., perceptual spatial resolution) and view-related image quality factors (e.g., focus cues) are significantly dependent on the characteristics of the human visual system. Retinal image formation models enable rigorous characterization and subsequently efficient design of light field and holographic displays. In this chapter the ray-based near-eye light field and wave-based near-eye holographic displays are reviewed, and the corresponding retinal image formation models are discussed. In particular, most of the discussion is devoted to characterization of the perceptual spatial resolution and focus cues. acceptedVersion

In this chapter, we motivate the use of densely-sampled light fields as the representation which can bring the required density of light rays for the correct recreation of 3D visual cues such as focus and continuous parallax and can serve as an intermediary between light field sensing and light field display. We consider the problem of reconstructing such a representation from few camera views and approach it in a sparsification framework. More specifically, we demonstrate that the light field is well structured in the set of so-called epipolar images and can be sparsely represented by a dictionary of directional and multi-scale atoms called shearlets. We present the corresponding regularization method, along with its main algorithm and speed-accelerating modifications. Finally, we illustrate its applicability for the cases of holographic stereograms and light field compression. acceptedVersion

ITER's working environment is characterized by extreme conditions, that deem maintenance and inspection tasks to be carried out through remote handling. 3D Node is a hardware/software module that extracts critical information from the remote environment during fine alignment tasks using an eye-in-hand camera system and updates the models behind the virtual reality-based remote handling platform. In this work we develop a retro-reflective marker-based version of 3D Node that estimates the pose of a planar target, the knuckle of the cassette locking system, using the markers attached to its surface. We demonstrate a pin-tool insertion task using these methods. Results show that our approach works reliably with a single low-resolution camera and outperforms the previously researched stereo depth estimation based approaches. We conclude that retro-reflective marker-based tracking has the potential to be a key enabler for remote handling operations in ITER., Comment: Accepted Version

The image-based rendering approach using Shearlet Transform (ST) is one of the state-of-the-art Densely-Sampled Light Field (DSLF) reconstruction methods. It reconstructs Epipolar-Plane Images (EPIs) in image domain via an iterative regularization algorithm restoring their coefficients in shearlet domain. Consequently, the ST method tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel self-supervised DSLF reconstruction method, CycleST, which applies ST and cycle consistency to DSLF reconstruction. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI reconstruction and cycle consistency losses. Besides, CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges ($\leqslant$ 8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16 - 32 pixels) from two challenging real-world light field datasets demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ~ 9x speedup over ST, at least.

We present a fast and accurate method for dense depth reconstruction from sparsely sampled light fields obtained using a synchronized camera array. In our method, the source images are over-segmented into non-overlapping compact superpixels that are used as basic data units for depth estimation and refinement. Superpixel representation provides a desirable reduction in the computational cost while preserving the image geometry with respect to the object contours. Each superpixel is modeled as a plane in the image space, allowing depth values to vary smoothly within the superpixel area. Initial depth maps, which are obtained by plane sweeping, are iteratively refined by propagating good correspondences within an image. To ensure the fast convergence of the iterative optimization process, we employ a highly parallel propagation scheme that operates on all the superpixels of all the images at once, making full use of the parallel graphics hardware. A few optimization iterations of the energy function incorporating superpixel-wise smoothness and geometric consistency constraints allows to recover depth with high accuracy in textured and textureless regions as well as areas with occlusions, producing dense globally consistent depth maps. We demonstrate that while the depth reconstruction takes about a second per full high-definition view, the accuracy of the obtained depth maps is comparable with the state-of-the-art results., Comment: 15 pages, 15 figures

We consider the problem of reconstructing densely sampled light field (DSLF) from sparse camera views. In our previous work, the DSLF has been reconstructed by processing epipolar-plane images (EPI) employing sparse regularization in shearlet transform domain. With the aim to avoid redundant processing and reduce the overall reconstruction time, in this article we propose algorithm modifications in three directions. First, we modify the basic algorithm by offering a faster and more stable iterative procedure. Second, we elaborate on the proper use of color redundancy by studying the effect of reconstruction of an average intensity channel and its use as a guiding mode for colorizing the three color channels. Third, we explore similarities between EPIs by their grouping and joint processing or by effective decorrelation to get an initial estimate for the basic iterative procedure. We are specifically interested in GPU-based computations allowing an efficient implementation of the shearlet transform. We quantify our three main approaches to accelerated processing over a wide collection of horizontal- as well as full-parallax datasets. acceptedVersion

Light field 3D displays are expected to play an important role as terminal devices, visualizing 3D objects apparently floating in the air, or letting viewers see through a window with a scene behind it. Currently, there are neither methods nor practical tools to quantify light field display's effective resolution or the perceived quality of the presented imagery. Most 3D displays are simply characterized by the total number of pixels or light rays; however this number does not properly characterize the distribution of the emitted light rays, nor the level of detail that the display can visualize properly. This paper presents methods to measure the spatial (i.e. 2D equivalent) and angular (i.e. directional) resolution of a given light-field display. The frequency domain analysis of recorded test patterns gives the spatial resolution limit of the display under test, while angular resolution is determined by the display's ability to pass through patterns of uniform, angle dependant patterns. It also presents a subjective experiment to corroborate the objectively measured spatial resolution. acceptedVersion

Pointwise intensity-based algorithms are the most popular algorithms in dynamic laser speckle measurement of physical or biological activity. The output of this measurement is a two-dimensional map which qualitatively separates regions of higher or lower activity. In the paper, we have proposed filtering of activity maps to enhance visualization and to enable quantitative determination of activity time scales. As a first step, we have proved that the severe spatial fluctuations within the map resemble a signal-dependent noise. As a second step, we have illustrated implementation of the proposed idea by applying filters to non-normalized and normalized activity estimates derived from synthetic and experimental data. Statistical behavior of the estimates has been analyzed to choose the filter parameters, and substantial narrowing of the probability density functions of the estimates has been achieved after the filtering. The filtered maps exhibit an improved contrast and allowed for quantitative description of activity. publishedVersion

Light fields representing Lambertian scenes are characterized by specific epipolar plane images, which support in Fourier domain is limited by the minimum and maximum scene depth. This feature of the light field has been taken into account when designing the shearlet-based light field reconstruction algorithm. However, this feature does not hold for the case of non-Lambertian scenes. Therefore, the original algorithm performs sub-optimally for such scenes. In this work, we extend the shearlet based reconstruction algorithm to enable light fields reconstruction for scenes composed of surfaces with non-Lambertian properties. Instead of assuming Lambertianity and the corresponding Fourier-domain support, we estimate the real Fourier-domain support of the given light field of a non-Lambertian scene. Based on the estimated support, we adapt the Shearlet frame to properly span that support. The results demonstrate that the new method yields a faithful reconstruction of densely sampled light fields of non-Lambertian scenes from a small number of captured views. acceptedVersion

We propose simple and efficient method that produces content-adaptive superpixels, i.e. smaller segments in content-dense areas and larger segments in content-sparse areas. Previous adaptive methods distribute superpixels over the image according to image content. In contrast, we transform the image itself to redistribute the content density uniformly across the image area. This transformation is guided by a significance map, which characterizes the ‘importance’ of each pixel. Arbitrary superpixel algorithm can be utilized to segment the transformed image into regular superpixels, providing a suitable representation for subsequent tasks. Regular superpixels in the transformed image induce content-adaptive superpixels in the original image facilitating the improved segmentation accuracy. acceptedVersion

Calibration of multiple cameras is a critical step in most vision enhancement systems. Target-based calibration approaches are known to provide accurate and stable results. However, they require manually performed capture procedures. This paper presents a generalization of a widely used single-camera target-based calibration algorithm to the case of n cameras. In order to obtain fully repeatable results, we propose the elimination of the manual capture step using a programmable robotic arm. Furthermore, we investigate the use of the position feedback provided by the robot. This is done specifically for the case of calibrating cameras without assumptions on their positions and overlapping of their fields of view. Results show that automatically captured images provide more accurate calibration results than the classical approach. Additionally, calibration of fully non-overlapping setups is made possible through our approach. acceptedVersion

Shearlet Transform (ST) is one of the most effective methods for Densely-Sampled Light Field (DSLF) reconstruction from a Sparsely-Sampled Light Field (SSLF). However, ST requires a precise disparity estimation of the SSLF. To this end, in this paper a state-of-the-art optical flow method, i.e. PWC-Net, is employed to estimate bidirectional disparity maps between neighboring views in the SSLF. Moreover, to take full advantage of optical flow and ST for DSLF reconstruction, a novel learning-based method, referred to as Flow-Assisted Shearlet Transform (FAST), is proposed in this paper. Specifically, FAST consists of two deep convolutional neural networks, i.e. disparity refinement network and view synthesis network, which fully leverage the disparity information to synthesize novel views via warping and blending and to improve the novel view synthesis performance of ST. Experimental results demonstrate the superiority of the proposed FAST method over the other state-of-the-art DSLF reconstruction methods on nine challenging real-world SSLF sub-datasets with large disparity ranges (up to 26 pixels). acceptedVersion

We present a learning-based optimization framework for depth of field extension, combining rigorous modeling of coded aperture imaging system and convolutional neural network based deblurring. The coded mask discretization is defined for desired depth range using wave optics based imaging model. Such approach significantly decreases the number of parameters to be optimized and increases the convergence speed of the network. We verify the proposed algorithm in different scenarios achieving superior or comparable performance with respect to existing methods. acceptedVersion

Shearlet Transform (ST) is one of the most effective approaches for light field reconstruction from Sparsely-Sampled Light Fields (SSLFs). This demo paper presents a comprehensive implementation of ST for light field reconstruction using one of the most popular machine learning libraries, i.e. Tensor Flow. The flexible architecture of TensorFlow allows for the easy deployment of ST across different platforms (CPUs, GPUs, TPUs) running varying operating systems with high efficiency and accuracy. acceptedVersion

Shearlet Transform (ST) is one of the most effective algorithms for the Densely-Sampled Light Field (DSLF) reconstruction from a Sparsely-Sampled Light Field (SSLF) with a large disparity range. However, ST requires a precise estimation of the disparity range of the SSLF in order to design a shearlet system with decent scales and to pre-shear the sparsely-sampled Epipolar-Plane Images (EPIs) of the SSLF. To overcome this limitation, a novel coarse-to-fine DSLF reconstruction method, referred to as Mask-Accelerated Shearlet Transform (MAST), is proposed in this paper. Specifically, a state-of-the-art learning-based optical flow method, FlowNet2, is employed to estimate the disparities of a SSLF. The estimated disparities are then utilized to roughly estimate the densely-sampled EPIs for the sparsely-sampled EPIs of the SSLF. Finally, an elaborately-designed soft mask for a coarsely-inpainted EPI is exploited to perform an iterative refinement on this EPI. Experimental results on nine challenging horizontal-parallax real-world SSLF datasets with large disparity ranges (up to 35 pixels) demonstrate the effectiveness and efficiency of the proposed method over the other state-of-the-art approaches. acceptedVersion

The performance of a light field reconstruction algorithm is typically affected by the disparity range of the input Sparsely-Sampled Light Field (SSLF). This paper finds that (i) one of the state-of-the-art video frame interpolation methods, i.e. adaptive Separable Convolution (SepConv), is especially effective for the light field reconstruction on a SSLF with a small disparity range ( 20 pixels). Therefore, to make full use of both methods to solve the challenging light field reconstruction problem on SSLFs with moderate and large disparity ranges, a novel method, referred to as Interpolation-Enhanced Shearlet Transform (IEST), is proposed by incorporating these two approaches in a coarse-to-fine manner. Specifically, ST is employed to give a coarse estimation for the target light field, which is then refined by SepConv to improve the reconstruction quality of parallax views involving small disparity ranges. Experimental results show that IEST outperforms the other state-of-the-art light field reconstruction methods on nine challenging horizontalparallax evaluation SSLF datasets of different real-world scenes with moderate and large disparity ranges. acceptedVersion

The International Thermonuclear Experimental Reactor (ITER) maintenance is performed by means of remote handling (RH) systems and with aid of user interfaces such as haptic and joystick devices, virtual reality (VR) systems, and camera views. Many RH operations involving RH equipment, such as robotic manipulator arms, require millimeter accuracy, but camera views are often occluded or of poor quality, and might be unavailable during sensitive steps that require accurate, close-up views. Moreover, the VR system may not reflect the current scene accurately, as physical conditions may have changed under the harsh environment. The purpose of this research was to prototype and evaluate a novel software system, called 3D Node, that locates and detects the position and orientation of a piece of RH equipment or reactor element with respect to a stereo camera pair. The detection information is utilized to adjust the motion trajectories of a robotic manipulator arm. The 3D Node features stereo-camera calibration, target depth mapping, target position and orientation detection, and online target tracking. This paper reports on the 3D Node demonstration on the ITER Divertor RH use case and discusses the system applicability to other ITER RH systems.

Remote operation of robot manipulators plays a crucial role in various safety-critical application areas such as forestry, mining, surveillance, etc. The capture and display of visual information of the real operation environment in an information-rich way is one of the key factors in achieving effective and robust teleoperation capability. In this paper, through subjective experiments, we comparatively investigate the roles of monocular, i.e. 2D, and stereo capture and display systems in the context of teleoperation of a robot arm. In particular, the positioning task is considered and, for both monocular and stereoscopic cases, two different modes of capture are implemented, which are static capture setup separately located from the robot arm and dynamic capture setup positioned at the tip of the robot arm. The experiments, conducted on 10 subjects (aged 24-33), show that stereo vision systems enable significant increase in accuracy of positioning compared to conventional 2D capture and display cases. In average, the tasks are completed with highest accuracy in the case of dynamic stereo capture setup. publishedVersion

We present a computational imaging approach, combining a phase-coded computational camera with a corresponding CNN-based deblurring network, that enables extended depth of field images. The simulations demonstrate promising results achieving significant depth of field extension. publishedVersion

3D imaging is important for enabling autonomous operation of smart mobile work machines. Different sensing techniques have different characteristics, which affect the choice of sensors for each application. We study the depth resolution of three different active imaging technologies and stereoscopic depth estimation with multiple different baselines. We test the effect of depth measurement abilities in measuring average depth, angles between planar structures and diameter from logs. Active sensors show their robustness, while stereoscopic depth estimation follows the error behavior expected from the theory. However, in practice we find that the effect of poor depth resolution can be significantly reduced by averaging over multiple measurements from different points and using more sophisticated stereo processing.

This chapter discusses the topic of emerging light field displays from a signal processing perspective. Light field displays are defined as devices which deliver continuous parallax along with the focus and binocular visual cues acting together in rivalry-free manner. In order to ensure such functionality, one has to deal with the light field, conceptualized by the plenoptic function and its adequate parametrization, sampling and reconstruction. The light field basics and the corresponding display technologies are overviewed in order to address the fundamental problems of analyzing light field displays as signal processing channels, and of capturing and representing light field visual content for driving such displays. Spectral analysis of multidimensional sampling operators is utilized to profile the displays in question, and modern sparsification approaches are employed to develop methods for high-quality light field reconstruction and rendering. acceptedVersion

The simplicity of the holographic stereogram (HS) makes it an attractive option in comparison to the more complex coherent computer generated hologram (CGH) methods. The cost of its simplicity is that the HS cannot accurately reconstruct deep scenes due to the lack of correct accommodation cues. The exact nature of the accommodation cues present in HSs, however, has not been investigated. In this paper, we provide analysis of the relation between the hologram sampling properties and the perceived accommodation response. The HS can be considered as a generator of a discrete light field (LF) and can thus be examined by considering the light ray oriented nature of the hologram diffracted light. We further support the analysis by employing a numerical reconstruction tool simulating the viewing process of the human eye. The simulation results demonstrate that HSs can provide accommodation cues depending on the choice of hologram segmentation size. It is further demonstrated that the accommodation response can be enhanced at the expense of loss in perceived spatial resolution. acceptedVersion

We present an accurate model of integral imaging display based on wave optics. The model enables accurate characterization of the display through simulated perceived images by the human visual system. Thus, it is useful to investigate the capabilities of the display in terms of various quality factors such as depth of field and resolution, as well as delivering visual cues such as focus. Furthermore, due to the adopted wave optics formalism, simulation and analysis of more advanced techniques such as wavefront coding for increased depth of field are also possible.