Search

The new event cameras are now widely used in many computer vision applications. Their high raw data bitrate levels require a more efficient fixed-length representation for low-bandwidth transmission from the event sensor to the processing chip. A novel low-complexity lossless compression framework is proposed for encoding the synchronous event frames (EFs) by introducing a novel memory-efficient fixed-length representation suitable for hardware implementation in the very-low-power (VLP) event-processing chip. A first contribution proposes an improved representation of the ternary frames using pixel-group frame partitioning and symbol remapping. Another contribution proposes a novel low-complexity memory-efficient fixed-length representation using multi-level lookup tables (LUTs). Complex experimental analysis is performed using a set of group-size configurations. For very-large group-size configurations, an improved representation is proposed using a mask-LUT structure. The experimental evaluation on a public dataset demonstrates that the proposed fixed-length coding framework provides at least two times the compression ratio relative to the raw EF representation and a close performance compared with variable-length video coding standards and variable-length state-of-the-art image codecs for lossless compression of ternary EFs generated at frequencies bellow one KHz. To our knowledge, the paper is the first to introduce a low-complexity memory-efficient fixed-length representation for lossless compression of synchronous EFs, suitable for integration into a VLP event-processing chip.

In this paper, a novel approach to encode lenslet (LL) images is proposed. The method departs from traditional block-based coding structures and employs a hexagonal-shaped pixel cluster, called macro-pixel, as an elementary coding unit. A novel prediction mode based on dictionary learning is proposed, whereby macro-pixels are represented by a sparse linear combination of atoms from a generic dictionary. Additionally, an optimized linear prediction mode and a directional prediction mode specifically designed for macro-pixels are proposed. Rate-distortion optimization is utilized to select the best intra prediction mode for each macro-pixel. Experimental results on the light field image data set show that the proposed coding system outperforms HEVC and the state-of-the-art in LL image coding with an average peak signal to noise ratio gain of 3.33 and 1.41 dB, respectively, and with rate savings of 67.13% and 34.30%, respectively.

The Letter introduces a novel filtering method based on convolutional neural networks (CNNs) for quality enhancement of light field (LF) images captured by a plenoptic camera and compressed using high-efficiency video coding (HEVC). The method takes advantage of the macro-pixel (MP) structure specific to the LF images and proposes a novel MP-wise filtering approach based on a novel deep neural network architecture. The proposed CNN-based method achieves an outstanding performance when HEVC is employed without its in-loop filters. Experimental results show high luminance-peak signal-to-noise ratio (Y-PSNR) gains and average Y-Bjontegaard delta (BD)-rate savings of 25.6 % over HEVC on a large data set.

The paper proposes a novel low-complexity Convolutional Neural Network (CNN) architecture for block-wise angular intra-prediction in lossless video coding. The proposed CNN architecture is designed based on an efficient patch processing layer structure. The proposed CNN-based prediction method is employed to process an input patch containing the causal neighborhood of the current block in order to directly generate the predicted block. The trained models are integrated in the HEVC video coding standard to perform CNN-based angular intra-prediction and to compete with the conventional HEVC prediction. The proposed CNN architecture contains a reduced number of parameters equivalent to only 37% of that of the state-of-the-art reference CNN architecture. Experimental results show that the inference runtime is also reduced by around 5.5% compared to that of the reference method. At the same time, the proposed coding systems yield 83% to 91% of the compression performance of the reference method. The results demonstrate the potential of structural and complexity optimizations in CNN-based intra-prediction for lossless HEVC.

A novel deep-learning-based depth estimation method for light field images is introduced. The proposed method employs a novel neural network design to estimate the disparity of each pixel based on block patches extracted from epipolar plane images. The network output is further refined based on filtering and denoising algorithms. Experimental results demonstrate an average improvement of 34.35 % in root mean squared error (RMSE) and 49.44 % in mean squared error over machine learning-based state-of-the-art methods.

The paper proposes a novel frame-wise view synthesis method based on convolutional neural networks (CNNs) for wide-baseline light field (LF) camera arrays. A novel neural network architecture that follows a multi-resolution processing paradigm is employed to synthesize an entire view. A novel loss function formulation based on the structural similarity index (SSIM) is proposed. A wide-baseline LF image dataset is generated and employed to train the proposed deep model. The proposed method synthesizes each subaperture image (SAI) from a LF image based on corresponding SAIs from two reference LF images. Experimental results show that the proposed method yields promising results with an average PSNR and SSIM of 34.71 dB and 0.9673 respectively for wide baselines.

This paper proposes a novel approach for lossless coding of light field (LF) images based on a macro-pixel (MP) synthesis technique which synthesizes the entire LF image in one step. The reference views used in the synthesis process are selected based on four different view configurations and define the reference LF image. This image is stored as an array of reference MPs which collect one pixel from each reference view, being losslessly encoded as a base layer. A first contribution focuses on a novel network design for view synthesis which synthesizes the entire LF image as an array of synthesized MPs. A second contribution proposes a network model for coding which computes the MP prediction used for lossless encoding of the remaining views as an enhancement layer. Synthesis results show an average distortion of 29.82 dB based on four reference views and up to 36.19 dB based on 25 reference views. Compression results show an average improvement of 29.9% over the traditional lossless image codecs and 9.1% over the state-of-the-art.

The paper introduces a novel macro-pixel prediction method based on Convolutional Neural Networks (CNN) for lossless compression of light field images. In the proposed method, each macro-pixel is predicted based on a volume of macro-pixels from its immediate causal neighborhood. The proposed deep neural network operates on these macro-pixel volumes and provides accurate macro-pixel prediction in light field images. The resulting macro-pixel residuals are encoded by a reference codec built based on the {CALIC} codec. A context modeling method for light field images is proposed. Experimental results on a large light field image dataset show that the proposed prediction method systematically and substantially outperforms state-of-the-art predictors. To our knowledge, the paper is the first to introduce deep-learning based prediction of macro-pixels, enabling efficient lossless compression of light field images.

The paper introduces a novel \(L_\infty \)-constrained compression method for depth maps. The proposed method performs depth segmentation and depth prediction in each segment, encoding the resulting information as a base layer. The depth residuals are modeled using a Two-Sided Geometric Distribution, and distortion and entropy models for the quantized residuals are derived based on such distributions. A set of optimal quantizers is determined to ensure a fix rate budget at a minimum \(L_\infty \) distortion. A fixed-rate \(L_\infty \) codec design performing context-based entropy coding of the quantized residuals is proposed, which is able to efficiently meet user constraints on rate or distortion. Additionally, a scalable \(L_\infty \) codec extension is proposed, which enables encoding the quantized residuals in a number of enhancement layers. The experimental results show that the proposed \(L_\infty \) coding approach substantially outperforms the \(L_\infty \) coding extension of the state-of-the-art CALIC method.

A novel residual-error prediction method based on deep learning with application in lossless image compression is introduced. The proposed method employs machine learning tools to minimise the residual error of the employed prediction tools. Experimental results demonstrate average bitrate savings of 32% over the state-of-the-art in lossless image compression. To the best of the authors’ knowledge, this Letter is the first to propose a deep-learning based method for residual-error prediction.

The paper proposes a novel prediction paradigm in image coding based on Convolutional Neural Networks (CNN). A deep neural network is designed to provide accurate pixel-wise prediction based on a causal neighbourhood. The proposed CNN prediction method is trained on the high-activity areas in the image and it is incorporated in a lossless compression system for high-resolution photographic images. The system uses the proposed CNN-based prediction paradigm as well as LOCO-I, whereby the predictor selection is performed using a local entropy-based descriptor. The prediction errors are encoded using a CALIC-based reference codec. The experimental results show a good performance for the proposed prediction scheme compared to state-of-the-art predictors. To our knowledge, the paper is the first to introduce CNN-based prediction in image coding, and demonstrates the potential offered by machine learning methods in coding applications.

The paper proposes a method for lossless compression of subaperture image stacks obtained by rectifying light-field images captured by a plenoptic camera. We exploit the similarities between two subaperture images using a predictive coding algorithm, where the current view is predicted from one reference view. Context modeling is the main technique used to reduce the image file size. A suitable image segmentation and a template context are used by the context tree algorithm for encoding up to the smallest detail in each subaperture image. Entropy coding is configured by a residual analysis module. The results show improved performance compared to the state-of-the-art encoders.

The paper proposes a novel block-wise prediction paradigm based on Convolutional Neural Networks (CNNs) for lossless video coding. A deep neural network model which follows a multi-resolution design is employed for block-wise prediction. Several contributions are proposed to improve neural network training. A first contribution proposes a novel loss function formulation for an efficient network training based on a new approach for patch selection. Another contribution consists in replacing all HEVC-based angular intra-prediction modes with a CNN-based intra-prediction method, where each angular prediction mode is complemented by a CNN-based prediction mode using a specifically trained model. Another contribution consists in an efficient adaptation of the CNN-based intra-prediction residual for lossless video coding. Experimental results on standard test sequences show that the proposed coding system outperforms the HEVC standard with an average bitrate improvement of around 5%. To our knowledge, the paper is the first to replace all the traditional HEVC-based angular intra-prediction modes with an intra-prediction method based on modern Machine Learning techniques for lossless video coding applications.

In this thesis, the problem of depth-map image compression is treated. The compilation of articles included in the thesis provides methodological contributions in the fields of lossless and lossy compression of depth-map images.The first group of methods addresses the lossless compression problem. The introduced methods are using the approach of representing the depth-map image in terms of regions and contours. In the depth-map image, a segmentation defines the regions, by grouping pixels having similar properties, and separates them using (region) contours. The depth-map image is encoded by the contours and the auxiliary information needed to reconstruct the depth values in each region.One way of encoding the contours is to describe them using two matrices of horizontal and vertical contour edges. The matrices are encoded using template context coding where each context tree is optimally pruned. In certain contexts, the contour edges are found deterministically using only the currently available information. Another way of encoding the contours is to describe them as a sequence of contour segments. Each such segment is defined by an anchor (starting) point and a string of contour edges, equivalent to a string of chain-code symbols. Here we propose efficient ways to select and encode the anchor points and to generate contour segments by using a contour crossing point analysis and by imposing rules that help in minimizing the number of anchor points.The regions are reconstructed at the decoder using predictive coding or the piecewise constant model representation. In the first approach, the large constant regions are found and one depth value is encoded for each such region. For the rest of the image, suitable regions are generated by constraining the local variation of the depth level from one pixel to another. The nonlinear predictors selected specifically for each region are combining the results of several linear predictors, each fitting optimally a subset of pixels belonging to the local neighborhood. In the second approach, the depth value of a given region is encoded using the depth values of the neighboring regions already encoded. The natural smoothness of the depth variation and the mutual exclusiveness of the values in neighboring regions are exploited to efficiently predict and encode the current region's depth value.The second group of methods is studying the lossy compression problem. In a first contribution, different segmentations are generated by varying the threshold for the depth local variability. A lossy depth-map image is obtained for each segmentation and is encoded based on predictive coding, quantization and context tree coding. In another contribution, the lossy versions of one image are created either by successively merging the constant regions of the original image, or by iteratively splitting the regions of a template image using horizontal or vertical line segments. Merging and splitting decisions are greedily taken, according to the best slope towards the next point in the rate-distortion curve. An entropy coding algorithm is used to encode each image.We propose also a progressive coding method for coding the sequence of lossy versions of a depth-map image. The bitstream is encoded so that any lossy version of the original image is generated, starting from a very low resolution up to lossless reconstruction. The partitions of the lossy versions into regions are assumed to be nested so that a higher resolution image is obtained by splitting some regions of a lower resolution image. A current image in the sequence is encoded using the a priori information from a previously encoded image: the anchor points are encoded relative to the already encoded contour points; the depth information of the newly resulting regions is recovered using the depth value of the parent region.As a final contribution, the dissertation includes a study of the parameterization of planar models. The quantized heights at three-pixel locations are used to compute the optimal plane for each region. The three-pixel locations are selected so that the distortion due to the approximation of the plane over the region is minimized. The planar model and the piecewise constant model are competing in the merging process, where the two regions to be merged are those ensuring the optimal slope in the rate-distortion curve.

In this paper we propose an extension of our lossy-to-lossless progressive coding method by placing the planar model in a competition with the piecewise constant model during the region reconstruction stage of the algorithm. A sequence of lossy images is generated using an hierarchical segmentation, of the initial image, based on region merging. The progressive coding method is able to compress this sequence of images by encoding the elements that represent the differences between two consecutive images. The method is splitting some regions from the current image segmentation using an encoded set of contours, and it is defining a set of new regions, which are reconstructed using either the piecewise constant model or the planar model. An efficient solution is proposed for encoding the model parameters in a progressive way. Results show an improvement of 3 – 4 dB compared to the baseline method based only on constant regions, and for a wide range it achieves almost similar results with the non-progressive methods.

A progressive coding method is proposed for depth-map images, where the bitstream is encoded so that one can generate many lossy versions of the original, encompassing a wide range, from very low resolution up to lossless reconstruction. The partitions into regions of the lossy versions are assumed to be nested, so that a higher resolution image is obtained by splitting some regions of a lower resolution image. The encoder transmits to the decoder information about which regions to split, the extra contour to be added for obtaining the shapes of the more refined regions, and the extra depth values needed inside each new region. The efficient encoding of the anchor points in the progressive scenario, relative to the contour points already encoded, and the depth information recovery, are the main contributions of this paper. The progressive bitstream produced by the proposed method scales well over the whole range of rates, from low rates to lossless, reaching a performance close to that of the non-progressive methods.