Search

This thesis investigates overcomplete decomposition of imagery for the purpose of compression. Overcomplete decomposition isan ill-posed inverse problem. Existing techniques impose prior information to the problem, generally in the form of sparsity orsmoothness constraints, to successfully decompose the original content. For the purpose of compression, it is important that theimposed constraints closely measure the total coding cost of the decomposed components. We propose a rate-distortion (R-D)driven optimization cost function for the problem of decomposition which not only improves the compression performance, butalso provides a meaningful and tractable procedure to separate information. The basic idea comes from the intuition that byincorporating efficient compression models, we are indeed choosing a better prior model for the underlying statistical problem.Applying this concept to the problem of decomposition, our approach penalizes unlikely estimates within a local region which isdifferent from rigid lp norms. We consider the diversity of components to be decomposed and choose distinct and differentsparsifying transforms for each component in the problem formulation. We employ the proposed decomposition approach to twoapplications. First, we develop a framework to decompose low frame rate video sequences into multiplicative illumination fieldsand illumination compensated residuals. We propose an illumination adaptive temporal transform which incorporates illuminationcompensation to lifting-based temporal wavelet transforms. We employ the R-D driven formulation to estimate full resolutionillumination fields, whose complexity is the subject of the optimization process. The R-D driven cost function is not convex but twoapproximations are explored, one being fully convex. A mesh-based illumination model is also developed to provide a point ofcomparison. The proposed method enables a highly scalable coding framework. Comparisons with weighted prediction in nonscala

Resolution scalability in images and video is performed in one of two ways: either by the method of Laplacian pyramids, or by an iterated subband filter bank such as the wavelet transform. In the case of the former, the method of Laplacian pyramids is an oversampled transform that does not offer true scalability because the cost of encoding increases for each additional level of resolution that is embedded.In contrast, the wavelet transform offers true scalability, with no overhead for arbitrary levels of resolution. However, the wavelet transform suffers from poor aliasing suppression capabilities, which is a fundamental limitation of the filter design of critically sampled subband transforms.In this thesis we present three novel transforms that are scalable, critically sampled, and offer superior aliasing suppression to the wavelet transform. We also provide comparison with state of the art in pyramid-based transforms.

This thesis explores graph-based regularization techniques for inverse problems in imaging and vision. Broadly speaking, the inverse problems we consider fall under one of two categories: image restoration, and surface reconstruction. The problem of image de-noising and de-blurring falls under the category of image restoration, while the problem of estimating optical flow or disparity falls under the category of surface reconstruction. In image restoration problems, the challenge is usually to smooth the image while preserving its features and texture at the same time, a seeming antithesis between the two competing goals. In surface reconstruction problems, the challenge is instead to reconstruct some piece-wise smooth representation of the underlying scene. In any case, we can use graphs to represent connections between pixels whose intensity values or motion vectors are related. Essentially, a graph defines an adaptive averaging filter whose filter kernel at a pixel is determined by the connection between the corresponding vertex and its neighbors on the graph. Traditionally, inverse problems are formulated using the Bayesian framework, the calculus of variations, or partial differential equations. However, all problems must be solved numerically using a digital computer, so they are ultimately discretised regardless of the original continuous-domain framework used to formulate them. Since graphs are algebraic constructs, we need not restrict ourselves to a grid-like topology. Formulating some given inverse problem directly on a graph is equivalent to defining an adaptive averaging filter for a given image. Later, we see that graphs are generally more useful if they are dense, i.e., there are many connections between pixels, and dynamic, i.e., the graph changes continually. The concepts of dense and dynamic graphs relate respectively to the idea of non-local operators and iteratively re-weighted optimization. An important aspect of graph Laplacians is that the gra

Resolution scalability in images and video is performed in one of two ways: either by the method of Laplacian pyramids, or by an iterated subband filter bank such as the wavelet transform. In the case of the former, the method of Laplacian pyramids is an oversampled transform that does not offer true scalability because the cost of encoding increases for each additional level of resolution that is embedded.In contrast, the wavelet transform offers true scalability, with no overhead for arbitrary levels of resolution. However, the wavelet transform suffers from poor aliasing suppression capabilities, which is a fundamental limitation of the filter design of critically sampled subband transforms.In this thesis we present three novel transforms that are scalable, critically sampled, and offer superior aliasing suppression to the wavelet transform. We also provide comparison with state of the art in pyramid-based transforms.

The spherical harmonic (SH) framework is a powerful representation that can be used to describe to a 3D soundfield. It decomposes waves propagating through space into a sum of infinitely many SH functions, a set of orthonormal basis functions over the sphere, and a set of source-dependent soundfield coefficients. The coefficients encode spatial information about the source and a wave propagation model. 3D soundfields can be captured or recorded using microphone arrays, manipulated or modified to enhance or reduce contributions from certain sources/locations, and reproduced using loudspeaker arrays to a degree of accuracy dependent on the array geometry. Other uses of the representation include SH beamforming, source localisation and isolation, and acoustic holography. This thesis focuses on the processes of coefficient extraction and some aspects of soundfield manipulation. Traditional extraction methods seek to gain a representation of all sources within the soundfield to a greater or lesser degree of accuracy. This thesis attempts to extract only the contributions from certain sources, spatially filtering the soundfield. The extraction process (involving discrete spatial sampling) introduces errors into the SH representation, in particular those of truncation error (TE) from using only a finite number of coefficients to reproduce a soundfield and spatial aliasing (SA) errors in the extracted coefficients. This thesis derives several closed form solutions for the TE (excluding SA) under various soundfield conditions, and shows the connection between the plane and spherical wave cases. It also investigates the patterns of SA caused by the regularised inverse (RI) and orthonormal extraction (OE) methods and observes the combined effects of SA and TE for a particular spherical microphone array. This thesis proposes a spatial Wiener filter (SWF) that makes use of infinite spatial order prior models of the propagation model, source power and location to reduce SA errors

This thesis investigates new motion anchoring strategies that are targeted at wavelet-based highly scalable video compression (WSVC). We depart from two practices that are deeply ingrained in existing video compression systems. Instead of the commonly used block motion, which has poor scalability attributes, we employ piecewise-smooth motion together with a highly scalable motion boundary description. The combination of this more “physical” motion description together with motion discontinuity information allows us to change the conventional strategy of anchoring motion at target frames to anchoring motion at reference frames, which improves motion inference across time.In the proposed reference-based motion anchoring strategies, motion fields are mapped from reference to target frames, where they serve as prediction references; during this mapping process, disoccluded regions are readily discovered. Observing that motion discontinuities displace with foreground objects, we propose motion-discontinuity driven motion mapping operations that handle traditionally challenging regions around moving objects. The reference-based motion anchoring exposes an intricate connection between temporal frame interpolation (TFI) and video compression. When employed in a compression system, all anchoring strategies explored in this thesis perform TFI once all residual information is quantized to zero at a given temporal level. The interpolation performance is evaluated on both natural and synthetic sequences, where we show favourable comparisons with state-of-the-art TFI schemes.We explore three reference-based motion anchoring strategies. In the first one, the motion anchoring is “flipped” with respect to a hierarchical B-frame structure. We develop an analytical model to determine the weights of the different spatio-temporal subbands, and assess the suitability and benefits of this reference-based WSVC for (highly scalable) video compression. Reduced motion coding cost and improved

Motion estimation has many applications in visual media processing. Most estimation schemes require a matching metric. One popular metric is mean square error in the image domain. In this case, information from different frequency bands and orientations are mixed together and treated the same way in the metric. Other metrics, such as Hadamard SAD, utilise decomposed image information as is, without further enhancement.This thesis proposes a series of matching metrics involving a split-enhance-fuse approach. The information in a video frame is first split into different frequency and/or orientation bands; each band is enhanced and matched independently. For each pixel location, scores calculated in detail bands are fused by summing across all scales and/or orientations. This diversifies the treatment of information in images. Several metrics of this type are analysed and compared to show that such diversity generates more accurate motion field.The decomposition consists of either a Laplacian pyramid or a directional pyramid. The enhancement involves adaptive thresholding and morphologically dilation. The thesis analyses these non-linear transforms and shows how the utility of high frequency bands can be improved for motion estimation, especially when motion is non-translational or when there is more than one motion model in a matching window, i.e., the window contains motion discontinuities. Mutual information inspired scores are calculated on enhanced bands, which is especially useful in the fusion process. An efficient implementation of the proposed method is described. The computational complexity is closely related to that of the disjoint block-based motion estimation by matching MSE. Finally the effects of edge preserving aggregation filters are investigated. A new edge-aware moving average is also proposed.

This work is concerned with the challenge of obtaining precise and reliable measurement of the vestibular anatomy in living human subjects at the limits of resolution of three dimensional (3D) computed tomography (CT). The structure and biomechanics of the vestibular system of the inner ear are responsible for providing the sensory information regarding the translational and angular motion of the head, used by the brain for stabilisation of posture and gaze. The measurement of this anatomical geometry is thus of the utmost importance for interpreting the function of the balance system in both normal and disease cases. In this project we investigate applying an active surface model for the recovery of the semicircular canals (SCC). The current literature provides very limited real results presented with respect to a ground truth. In contrast, we evaluate model performance using real clinical CT data with comparisons made to an objective ground truth acquired using micro-CT. At small sub-pixel scales the role of regularization is especially important, where the application of conventional active contour methods leads to non-optimal results. First of all, we study the influence of various curvature formulations, and show that conventional terms introduce a tendency to shrink or expand the solution. We proposed a new scale invariant smoothing term that eliminates this bias. Secondly, we show that the blind use of a gradient magnitude based energy performs poorly and we recognise the role played by the point spread function (PSF) interacting with small features. Thirdly, we address the influence of the PSF by devising an augmented active surface model by incorporating the PSF into the external energy. We also propose augmenting our model with further prior knowledge, that of reasonably constant underlying CT numbers.The final contribution of this thesis is a precise measurement of the SCC morphology in 34 human ears, combined with a detailed comparison and analysis withi

Video is considered one of the main applications of modern day's Internet. Despite its importance, the interactivity available from current implementations is limited to pause and random access to a set of predetermined access points. In this work, we propose a novel and innovative approach which provides considerably better interactivity and we coin the term JPEG2000-Based Scalable Interactive Video (JSIV) for it. JSIV relies on three main concepts: storing the video sequence as independent JPEG2000 frames to provide for quality and spatial resolution scalability, as well as temporal and spatial accessibility; prediction and conditional replenishment of precincts to exploit inter-frame redundancy; and loosely-coupled server and client policies. The concept of loosely-coupled client and server policies is central to JSIV. With these policies, the server optimally selects the number of quality layers for each precinct it transmits and decides on any side-information that needs to be transmitted while the client attempts to make most of the received (distorted) frames. In particular, the client decides which precincts are predicted and which are decoded from received data (or possibly filled with zeros in the absence of received data). Thus, in JSIV, a predicted frame typically has some of its precincts predicted from nearby frames while others are decoded from received intra-coded precincts; JSIV never uses frame differences or prediction residues.The philosophy behind these policies is that neither the server nor the client drives the video streaming interaction, but rather the server dynamically selects and sends the pieces that, it thinks, best serve the client needs and, in turn, the client makes most of the pieces of information it has. The JSIV paradigm postulates that if both the client and the server policies are intelligent enough and make reasonable decisions, then the decisions made by the server are likely to have the expected impact on the client's decis

Modeling the motion that occurs between frames of a video sequence is a key component of video coding applications. Typically it is not possible to represent the motion between frames by a single model and therefore a quad-tree structure is employed where smaller, variable size regions or blocks are allowed to take on separate motion models. Quad-tree structures however suffer from two fundamental forms of redundancy. First, quad-trees exhibit structural redundancy due to their inability to exploit the dependence between neighboring leaf nodes with different parents. The second form of redundancy is due to the quad-tree structure itself being limited to capture only horizontal and vertical edge discontinuities at dyadically related locations; this means that general discontinuities in the motion field, such as those caused by boundaries of moving objects, become difficult and expensive to model.In our work, we address the issue of structural redundancy by introducing leaf merging. We describe how the intuitively appealing leaf merging step can be incorporated into quad-tree motion representations for a range motion modeling contexts. In particular, the impact of rate-distortion (R-D) optimized merging for two motion coding schemes, these being spatially predictive coding, as used by H.264, and hierarchical coding, are considered. Our experimental results demonstrate that the merging step can provide significant gains in R-D performance for both the hierarchical and spatial prediction schemes. Hierarchical coding has the advantage that it offers scalable access to the motion information; however due to the redundancy it introduces hierarchical coding has not been traditionally pursued. Our work shows that much of this redundancy can be mitigated with the introduction of merging. To enable scalable decoding, we employ a merging scheme which ensures that the dependencies introduced via merging can be hierarchically decoded. Theoretical investigations confirm the inhere

Efficient methods for the calculation of radial influence are described and applied to two image processing problems, digital halftoning and mixed content image compression. The methods operate recursively on scanlines of image values, spreading intensity from scanline to scanline in proportions approximating a Cauchy distribution.For error diffusion halftoning, experiments show that this recursive scanline spreading provides an ideal pattern of distribution of error. Error diffusion using masks generated to provide this distribution of error alleviate error diffusion "worm" artifacts.The recursive scanline by scanline application of a spreading filter and a complementary filter can be used to reconstruct an image from its horizontal and vertical pixel difference values. When combined with the use of a downsampled image the reconstruction is robust to incomplete and quantized pixel difference data. Such gradient field integration methods are described in detail proceeding from representation of images by gradient values along contours through to a variety of efficient algorithms.Comparisons show that this form of gradient field integration by convolution provides reduced distortion compared to other high speed gradient integration methods. The reduced distortion can be attributed to success in approximating a radial pattern of influence. An approach to edge-based image compression is proposed using integration of gradient data along edge contours and regularly sampled low resolution image data. This edge-based image compression model is similar to previous sketch based image coding methods but allows a simple and efficient calculation of an edge-based approximation image.A low complexity implementation of this approach to compression is described. The implementation extracts and represents gradient data along edge contours as pixel differences and calculates an approximate image by performing integration of pixel difference data by scanline convolution. The implemen