Search

The new event cameras are now widely used in many computer vision applications. Their high raw data bitrate levels require a more efficient fixed-length representation for low-bandwidth transmission from the event sensor to the processing chip. A novel low-complexity lossless compression framework is proposed for encoding the synchronous event frames (EFs) by introducing a novel memory-efficient fixed-length representation suitable for hardware implementation in the very-low-power (VLP) event-processing chip. A first contribution proposes an improved representation of the ternary frames using pixel-group frame partitioning and symbol remapping. Another contribution proposes a novel low-complexity memory-efficient fixed-length representation using multi-level lookup tables (LUTs). Complex experimental analysis is performed using a set of group-size configurations. For very-large group-size configurations, an improved representation is proposed using a mask-LUT structure. The experimental evaluation on a public dataset demonstrates that the proposed fixed-length coding framework provides at least two times the compression ratio relative to the raw EF representation and a close performance compared with variable-length video coding standards and variable-length state-of-the-art image codecs for lossless compression of ternary EFs generated at frequencies bellow one KHz. To our knowledge, the paper is the first to introduce a low-complexity memory-efficient fixed-length representation for lossless compression of synchronous EFs, suitable for integration into a VLP event-processing chip.

The paper proposes a novel frame-wise filtering method based on Convolutional Neural Networks (CNNs) for enhancing the quality of HEVC decoded videos. A novel deep neural network architecture is proposed for post-filtering the entire intra-coded videos. A novel scheme utilizing frame-size patches is employed for training the network. The proposed method filters the luminance channel separately from the pair of chrominance channels. A novel patch generation paradigm is proposed where, for each color channel, the corresponding mode map is generated based on the HEVC intra-prediction mode index and block segmentation. The proposed CNN-based filtering method is an alternative to the traditional HEVC built-in in-loop filtering module for intra-coded frames. Experimental results on standard test sequences show that the proposed method outperforms the HEVC standard with average BD-rate savings of 11.1% and an average BD-PSNR improvement of 0.602 dB. The average relative improvement in $\Delta $ PSNR is around 105% at $QP = 42$ and around 85% at $QP = 32$ compared with state-of-the-art machine-learning-based methods.

In this paper, a novel approach to encode lenslet (LL) images is proposed. The method departs from traditional block-based coding structures and employs a hexagonal-shaped pixel cluster, called macro-pixel, as an elementary coding unit. A novel prediction mode based on dictionary learning is proposed, whereby macro-pixels are represented by a sparse linear combination of atoms from a generic dictionary. Additionally, an optimized linear prediction mode and a directional prediction mode specifically designed for macro-pixels are proposed. Rate-distortion optimization is utilized to select the best intra prediction mode for each macro-pixel. Experimental results on the light field image data set show that the proposed coding system outperforms HEVC and the state-of-the-art in LL image coding with an average peak signal to noise ratio gain of 3.33 and 1.41 dB, respectively, and with rate savings of 67.13% and 34.30%, respectively.

The Letter introduces a novel filtering method based on convolutional neural networks (CNNs) for quality enhancement of light field (LF) images captured by a plenoptic camera and compressed using high-efficiency video coding (HEVC). The method takes advantage of the macro-pixel (MP) structure specific to the LF images and proposes a novel MP-wise filtering approach based on a novel deep neural network architecture. The proposed CNN-based method achieves an outstanding performance when HEVC is employed without its in-loop filters. Experimental results show high luminance-peak signal-to-noise ratio (Y-PSNR) gains and average Y-Bjontegaard delta (BD)-rate savings of 25.6 % over HEVC on a large data set.

A novel deep-learning-based depth estimation method for light field images is introduced. The proposed method employs a novel neural network design to estimate the disparity of each pixel based on block patches extracted from epipolar plane images. The network output is further refined based on filtering and denoising algorithms. Experimental results demonstrate an average improvement of 34.35 % in root mean squared error (RMSE) and 49.44 % in mean squared error over machine learning-based state-of-the-art methods.

This paper proposes a novel approach for lossless coding of light field (LF) images based on a macro-pixel (MP) synthesis technique which synthesizes the entire LF image in one step. The reference views used in the synthesis process are selected based on four different view configurations and define the reference LF image. This image is stored as an array of reference MPs which collect one pixel from each reference view, being losslessly encoded as a base layer. A first contribution focuses on a novel network design for view synthesis which synthesizes the entire LF image as an array of synthesized MPs. A second contribution proposes a network model for coding which computes the MP prediction used for lossless encoding of the remaining views as an enhancement layer. Synthesis results show an average distortion of 29.82 dB based on four reference views and up to 36.19 dB based on 25 reference views. Compression results show an average improvement of 29.9% over the traditional lossless image codecs and 9.1% over the state-of-the-art.

A novel residual-error prediction method based on deep learning with application in lossless image compression is introduced. The proposed method employs machine learning tools to minimise the residual error of the employed prediction tools. Experimental results demonstrate average bitrate savings of 32% over the state-of-the-art in lossless image compression. To the best of the authors’ knowledge, this Letter is the first to propose a deep-learning based method for residual-error prediction.

The paper proposes a novel block-wise prediction paradigm based on Convolutional Neural Networks (CNNs) for lossless video coding. A deep neural network model which follows a multi-resolution design is employed for block-wise prediction. Several contributions are proposed to improve neural network training. A first contribution proposes a novel loss function formulation for an efficient network training based on a new approach for patch selection. Another contribution consists in replacing all HEVC-based angular intra-prediction modes with a CNN-based intra-prediction method, where each angular prediction mode is complemented by a CNN-based prediction mode using a specifically trained model. Another contribution consists in an efficient adaptation of the CNN-based intra-prediction residual for lossless video coding. Experimental results on standard test sequences show that the proposed coding system outperforms the HEVC standard with an average bitrate improvement of around 5%. To our knowledge, the paper is the first to replace all the traditional HEVC-based angular intra-prediction modes with an intra-prediction method based on modern Machine Learning techniques for lossless video coding applications.

In this thesis, the problem of depth-map image compression is treated. The compilation of articles included in the thesis provides methodological contributions in the fields of lossless and lossy compression of depth-map images.The first group of methods addresses the lossless compression problem. The introduced methods are using the approach of representing the depth-map image in terms of regions and contours. In the depth-map image, a segmentation defines the regions, by grouping pixels having similar properties, and separates them using (region) contours. The depth-map image is encoded by the contours and the auxiliary information needed to reconstruct the depth values in each region.One way of encoding the contours is to describe them using two matrices of horizontal and vertical contour edges. The matrices are encoded using template context coding where each context tree is optimally pruned. In certain contexts, the contour edges are found deterministically using only the currently available information. Another way of encoding the contours is to describe them as a sequence of contour segments. Each such segment is defined by an anchor (starting) point and a string of contour edges, equivalent to a string of chain-code symbols. Here we propose efficient ways to select and encode the anchor points and to generate contour segments by using a contour crossing point analysis and by imposing rules that help in minimizing the number of anchor points.The regions are reconstructed at the decoder using predictive coding or the piecewise constant model representation. In the first approach, the large constant regions are found and one depth value is encoded for each such region. For the rest of the image, suitable regions are generated by constraining the local variation of the depth level from one pixel to another. The nonlinear predictors selected specifically for each region are combining the results of several linear predictors, each fitting optimally a subset of pixels belonging to the local neighborhood. In the second approach, the depth value of a given region is encoded using the depth values of the neighboring regions already encoded. The natural smoothness of the depth variation and the mutual exclusiveness of the values in neighboring regions are exploited to efficiently predict and encode the current region's depth value.The second group of methods is studying the lossy compression problem. In a first contribution, different segmentations are generated by varying the threshold for the depth local variability. A lossy depth-map image is obtained for each segmentation and is encoded based on predictive coding, quantization and context tree coding. In another contribution, the lossy versions of one image are created either by successively merging the constant regions of the original image, or by iteratively splitting the regions of a template image using horizontal or vertical line segments. Merging and splitting decisions are greedily taken, according to the best slope towards the next point in the rate-distortion curve. An entropy coding algorithm is used to encode each image.We propose also a progressive coding method for coding the sequence of lossy versions of a depth-map image. The bitstream is encoded so that any lossy version of the original image is generated, starting from a very low resolution up to lossless reconstruction. The partitions of the lossy versions into regions are assumed to be nested so that a higher resolution image is obtained by splitting some regions of a lower resolution image. A current image in the sequence is encoded using the a priori information from a previously encoded image: the anchor points are encoded relative to the already encoded contour points; the depth information of the newly resulting regions is recovered using the depth value of the parent region.As a final contribution, the dissertation includes a study of the parameterization of planar models. The quantized heights at three-pixel locations are used to compute the optimal plane for each region. The three-pixel locations are selected so that the distortion due to the approximation of the plane over the region is minimized. The planar model and the piecewise constant model are competing in the merging process, where the two regions to be merged are those ensuring the optimal slope in the rate-distortion curve.

This paper introduces an efficient method for lossless compression of depth map images, using the representation of a depth image in terms of three entities: 1) the crack-edges; 2) the constant depth regions enclosed by them; and 3) the depth value over each region. The starting representation is identical with that used in a very efficient coder for palette images, the piecewise-constant image model coding, but the techniques used for coding the elements of the representation are more advanced and especially suitable for the type of redundancy present in depth images. Initially, the vertical and horizontal crack-edges separating the constant depth regions are transmitted by 2D context coding using optimally pruned context trees. Both the encoder and decoder can reconstruct the regions of constant depth from the transmitted crack-edge image. The depth value in a given region is encoded using the depth values of the neighboring regions already encoded, exploiting the natural smoothness of the depth variation, and the mutual exclusiveness of the values in neighboring regions. The encoding method is suitable for lossless compression of depth images, obtaining compression of about 10-65 times, and additionally can be used as the entropy coding stage for lossy depth compression.

Publication in the conference proceedings of EUSIPCO, Bucharest, Romania, 2012

In this paper, we propose a low complexity method for detection and tracking of potholes in video sequences taken by a camera placed inside a moving car. The region of interest for the detection of the potholes is selected as the image area where the road is observed with the highest resolution. A threshold-based algorithm generates a set of candidate regions. For each region the following features are extracted: its size, the regularity of the intensity surface, contrast with respect to background model, and the region's contour length and shape. The candidate regions are labeled as putative potholes by a decision tree according to these features, eliminating the false positives due to shadows of wayside objects. The putative potholes that are successfully tracked in consecutive frames are finally declared potholes. Experimental results with real video sequences show a good detection precision. acceptedVersion