Search

In this paper, we present a spatially adaptive method to reduce compression artifacts observed in block discrete cosine transform (DCT) based image/video compression standards. The method is based on the bilateral filter, which is very effective in denoising images without smoothing edges. When applied to reduce compression artifacts, the parameters of the bilateral filter should be chosen carefully to have a good performance. To avoid over-smoothing texture regions and to effectively eliminate blocking and ringing artifacts, in this paper, texture regions and block boundary discontinuities are first detected; these are then used to control/adapt the spatial and intensity parameters of the bilateral filter. Experiments show that the proposed method improves over the standard non-adaptive bilateral filter visually and quantitatively.

Most commercial digital cameras have three types of color sensors (for red, green, and blue channels) that are placed on a detector surface according to a specific pattern. At the location of each pixel only one color sample is taken, and the values of the other colors must be interpolated using neighboring samples. This color plane interpolation is known as demosaicing; it is one of the important tasks in a digital camera pipeline. If demosaicing is not performed appropriately, images suffer from highly visible color artifacts. In this paper we present a new demosaicing technique that uses inter-channel correlation effectively in an alternating-projections scheme. We have compared this technique with various state-of-the-art demosaicing techniques, and it outperforms all of them, both visually and in terms of mean square error.

Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing step to obtain a high-resolution image that is later passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose to transfer the super-resolution reconstruction from pixel domain to a lower dimensional face space. Such an approach has the advantage of a significant decrease in the computational complexity of the super-resolution reconstruction. The reconstruction algorithm no longer tries to obtain a visually improved high-quality image, but instead constructs the information required by the recognition system directly in the low dimensional domain without any unnecessary overhead. In addition, we show that face-space super-resolution is more robust to registration errors and noise than pixel-domain super-resolution because of the addition of model-based constraints.

Image demosaicing is a problem of interpolating full-resolution color images from so-called color-filter-array (CFA) samples. Among various CFA patterns, Bayer pattern has been the most popular choice and demosaicing of Bayer pattern has attracted renewed interest in recent years partially due to the increased availability of source codes/executables in response to the principle of "reproducible research". In this article, we provide a systematic survey of over seventy published works in this field since 1999 (complementary to previous reviews22, 67). Our review attempts to address important issues to demosaicing and identify fundamental differences among competing approaches. Our findings suggest most existing works belong to the class of sequential demosaicing - i.e., luminance channel is interpolated first and then chrominance channels are reconstructed based on recovered luminance information. We report our comparative study results with a collection of eleven competing algorithms whose source codes or executables are provided by the authors. Our comparison is performed on two data sets: Kodak PhotoCD (popular choice) and IMAX high-quality images (more challenging). While most existing demosaicing algorithms achieve good performance on the Kodak data set, their performance on the IMAX one (images with varying-hue and high-saturation edges) degrades significantly. Such observation suggests the importance of properly addressing the issue of mismatch between assumed model and observation data in demosaicing, which calls for further investigation on issues such as model validation, test data selection and performance evaluation.

The bilateral filter is a nonlinear filter that does spatial averaging without smoothing edges; it has shown to be an effective image denoising technique in addition to some other applications. There are two main contributions of this paper. First, we provide an empirical study of the optimal parameter selection for the bilateral filter in image denoising applications. Second, we present an extension of the bilateral filter: multi-resolution bilateral filter, where bilateral filtering is applied to low-frequency subbands of a signal decomposed using an orthogonal wavelet transform. Combined with wavelet thresholding, this new image denoising framework turns out to be very effective in eliminating noise in real noisy images. We provide experimental results with both simulated data and real data.

Spatial resolution of digital images are limited due to optical/sensor blurring and sensor site density. In single-chip digital cameras, the resolution is further degraded because such devices use a color filter array to capture only one spectral component at a pixel location. The process of estimating the missing two color values at each pixel location is known as demosaicking. Demosaicking methods usually exploit the correlation among color channels. When there are multiple images, it is possible not only to have better estimates of the missing color values but also to improve the spatial resolution further (using super-resolution reconstruction). Previously, we have proposed a demosaicking algorithm based on the projection onto convex sets (POCS) technique. In this paper, we improve the results of that algorithm adding a new constraint set based on the spatio-intensity neighborhood. We extend the algorithm to image sequences for multi-frame demosaicking and super resolution.

Image registration is a necessary step in a variety of computer vision applications. One of the recent focus areas in image registration is extracting and matching features that are invariant to affine transformation. This is critical in various applications, including 3D reconstruction and object recognition. In this paper, we present a feature-based image registration method that is robust to scaling and rotation. This is achieved by extracting and matching features in the log-polar domain, where rotation and scale correspond to translation. Registration parameters are then estimated by applying the RANSAC technique to the feature correspondences. The RANSAC technique provides a robust estimation even when there are moving objects within the scene. Experimental results with synthetic and real images are provided.

The process of reconstructing a high-resolution image from multiple low-resolution images is known as super-resolution reconstruction. One of the assumptions of current super-resolution reconstruction methods is that all images are captured with the same exposure time and aperture size. This is not necessarily true. In reality, cameras have limited dynamic range and a nonlinear response to the quantity of light received; also, camera settings might be adjusted automatically or manually to capture the desired portion of the scene's dynamic range. We propose a super-resolution algorithm based on an imaging model that includes a camera response function, exposure time, sensor noise and quantization error in addition to spatial blurring and sampling. The algorithm is based on Bayesian estimation. Initial experiments demonstrate the effectiveness of the algorithm.

In image retrieval applications one of the boundary-dependent approaches is matching contours with their polygonal representation. We introduce (1) a new polygonal shape representation, (2) an efficient algorithm to compute a unique representation of a polygon to handle orientation and (3) a matching method that is invariant to rigid and affine transformation. In the method, polygons are represented by a sequence of distance vectors ordered in a predefined cyclic way. Each vector is composed of two primitives which are radial distance from the centroid to a vertex and the following edge distance in a specified direction. Matching of polygons is achieved by bitwise comparison of their string code. The algorithm has a computational complexity of O(n log n), hence it has advantage for practical use.

Modern digital image processing requires powerful data compression algorithms to allow the data to be efficiently transferred from the host to end users (and back again). A typical 512times512 grayscale image of uncompressed data requires more than quarter of a million bytes. Current image compression standards like JPEG2000 and the FBI WSQ (wavelet scalar quantization) use wavelet transforms with quantization to compress still images, which reconstruct with high accuracy. This paper considers a number of popular 9/7 wavelet transform architectures. High level software models are developed for these transforms to validate their effectiveness. These software models are modified and evaluated as reversible integer wavelet lifting transforms. Further, using a virtual hardware design targeted to reconfigurable FPGA technology these transforms are implemented into a 2D discrete wavelet transform (DWT) image processor with DDR SDRAM operating at core speeds of 200+ MHz. Finally, our Matlab and Maple models perform the validation of wavelet lifting transforms

Super-resolution reconstruction is the process of reconstructing a high-resolution image from multiple low-resolution images. Most super-resolution reconstruction methods neglect camera response function, exposure time, white balancing, and external illumination changes. In this paper, we show how to extend traditional super-resolution reconstruction methods to handle these factors. We provide formulations for several different super-resolution reconstruction approaches. Experimental results are also included.

Considerable attention has been directed to the problem of producing high-resolution video and still images from multiple low-resolution images. This multiframe reconstruction, also known as super-resolution reconstruction, is beginning to be applied to compressed video. Super-resolution techniques that have been designed for raw (i.e., uncompressed) video may not be effective when applied to compressed video because they do not incorporate the compression process into their models. The compression process introduces quantization error, which is the dominant source of error in some cases. In this paper, we propose a stochastic framework where quantization information as well as other statistical information about additive noise and image prior can be utilized effectively.

An imaging system produces a degraded measurement of a real-valued quantity that varies temporally, spectrally, and spatially. When there are multiple measurements of the same scene, it is possible to combine the nonredundant information in those measurements and produce an improved image that has more information than any of the measurements does alone. This type of reconstruction is known as image fusion. Depending on the diversity of information in the measurements, it is possible to achieve temporal, spectral, spatial, and gray-scale improvements with an image fusion algorithm. In this paper, we are proposing an image fusion algorithm that produces an image of higher spatial and gray-scale information from multiple measurements. The algorithm estimates the spatial and illumination correlation between multiple measurements, and employs a set-theoretic reconstruction technique.

Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed that attempt to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing system to obtain a high resolution image that can later be passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose embedding the super-resolution algorithm into the face recognition system so that super-resolution is not performed in the pixel domain, but is instead performed in a reduced dimensional domain. The advantage of such an approach is a significant decrease in the computational complexity of the super-resolution algorithm because the algorithm no longer tries to construct a visually improved high quality image, but instead constructs the information required by the recognition algorithm directly in the lower dimensional domain without any unnecessary overhead.

Resolution enhancement for video sequences has always been an attractive application in multimedia signal processing. "Superresolution" methods, that combine non-redundant information from a set of low-resolution images, are beginning to be applied to the most popular video compression standard, MPEG. Bayesian approaches, which are very successful for raw video, largely fail for MPEG video, since they do not incorporate the compression process into their models. This compression process introduces quantization noise, which is comparable to the additive noise that is used in the Bayesian models. We present an analytical derivation that combines the quantization and additive noises in a stochastic framework for MPEG-compressed video. This is a general framework in the sense that different video acquisition models, source statistics, implementation techniques can be used with it.

The number of bits assigned to represent the color intensity at image pixels is usually referred as the bit depth. When the bit depth is not sufficient, images suffer from ridge-like structures known as false contours. Bit-depth limitations become important when low-contrast details are required, as in medical imaging, aerial/satellite photography, and high-quality scanning: applications. In this paper, we investigate a method for increasing bit depth. Specifically, we show that when a sequence of video frames is available, then it is possible to achieve a higher bit depth through a projections onto convex sets (POCS) based reconstruction method.

A major drawback of block-based still image or video compression methods at low rates are the visible block boundaries that are also known as blocking artifacts. Several methods have been proposed in the literature to reduce these artifacts for video sequences. However, most are simply adaptations of still image blocking artifact reduction methods, which do not exploit temporal information. We propose a novel multi-frame blocking artifact reduction method that incorporates temporal information effectively. This method uses the spatial correlations that exist between the successive frames to define constraint sets at multiple frames and provides a Projections Onto Convex Sets (POCS) solution. The proposed method operates solely on transform domain (DCT) data, and hence provides a solution that is compatible with the observed video. It does not need to make any spatial smoothness assumptions, which are typical with blocking artifact reduction algorithms for still images.

Multi-transmitter aperture synthesis increases the effective aperture in coherent imaging by shifting the backscattered speckle field across a physical aperture or set of apertures. Through proper arrangement of the transmitter locations, it is possible to obtain speckle fields with overlapping regions, which allows fast computation of optical aberrations from wavefront differences. In this paper, we present a method where Zernike polynomials are used to model the aberrations and high-order aberrations are estimated without the need to do phase unwrapping of the difference fronts.

The resolution of a diffraction-limited imaging system is inversely proportional to the aperture size. Instead of using a single large aperture, multiple small apertures are used to synthesize a large aperture. Such a multi-aperture system is modular, typically more reliable and less costly. On the other hand, a multi-aperture system requires phasing sub-apertures to within a fraction of a wavelength. So far in the literature, only the piston, tip, and tilt type of inter-aperture errors have been addressed. In this paper, we present an approach to correct for rotational and translational errors as well.