Search

There are domain-specific image classification problems such as facial emotion and house-number classifications, where the color information in the images may not be crucial for recognition. This motivates us to convert RGB images to gray-scale ones with a single Y channel to be fed into the pre-trained convolutional neural networks (CNN). Now, since the existing CNN models are pre-trained by three-channel color images, one can expect that some trained filters are more sensitive to colors than brightness. Therefore, adopting the single-channel gray-scale images as inputs, we can prune out some of the convolutional filters in the first layer of the pre-trained CNN. This first-layer pruning greatly facilitates the filter compression of the subsequent convolutional layers. Now, the pre-trained CNN with the compressed filters is fine-tuned with the single-channel images for a domain-specific dataset. Experimental results on the facial emotion and Street View House Numbers (SVHN) datasets show that we can achieve a significant compression of the pre-trained CNN filters by the proposed method. For example, compared with the fine-tuned VGG-16 model by color images, we can save 10.538 GFLOPs computations, while keeping the classification accuracy around 84% for the facial emotion RAF-DB dataset.

A pre-trained 2D CNN (Convolutional Neural Network) can be used for the spatial stream in the two-stream CNN structure for videos, treating the representative frame selected from the video as an input. However, the CNN for the temporal stream in the two-stream CNN needs training from scratch using the optical flow frames, which demands expensive computations. In this paper, we propose to adopt a pre-trained 2D CNN for the temporal stream to avoid the optical flow computations. Specifically, three RGB frames selected at three different times in the video sequence are converted into grayscale images and are assigned to three R(red), G(green), and B(blue) channels, respectively, to form a Stacked Grayscale 3-channel Image (SG3I). Then, the pre-trained 2D CNN is fine-tuned by SG3Is for the temporal stream CNN. Therefore, only pre-trained 2D CNNs are used for both spatial and temporal streams. To learn long-range temporal motions in videos, we can use multiple SG3Is by partitioning the video shot into sub-shots and a single SG3I is generated for each sub-shot. Experimental results show that our two-stream CNN with the proposed SG3Is is about 14.6 times faster than the first version of the two-stream CNN with the optical flow, and yet achieves a similar recognition accuracy for UCF-101 and a 5.7% better result for HMDB-51.

Most conventional fine-grained image recognitions are based on a two-stream model of object-level and part-level CNNs, where the part-level CNN is responsible for learning the object-parts and their spatial relationships. To train the part-level CNN, we first need to separate parts from an object. However, there exist sub-level objects with no distinctive and separable parts. In this paper, a multi-scale CNN with a baseline Object-level and multiple Part-level CNNs is proposed for the fine-grained image recognition with no separable object-parts. The basic idea to train different CNNs of the multi-scale CNNs is to adopt different scales in resizing the training images. That is, the training images are resized such that the entire object appears as much as possible for the Object-level CNN, while only a local part of the object is to be included for the Part-level CNN. This scale-specific image resizing approach requires a scale-controllable parameter in the image resizing process. In this paper, a scale-controllable parameter is introduced for the linear-scaling and random-cropping method. Also, a line-based image resizing method with a scale-controllable parameter is employed for the part-level CNNs. The proposed multi-scale CNN is applied to a food image classification, which belongs to a fine-grained classification problem with no separable object-parts. Experimental results on the public food image datasets show that the classification accuracy improves substantially when the predicted scores of the multi-scale CNN are fused together. This reveals that the object-level and part-level CNNs work harmoniously in differentiating subtle differences of the sub-level objects.

This paper deals with the problem of training convolutional neural networks (CNNs) with facial action units (AUs). In particular, we focus on the imbalance problem of the training datasets for facial emotion classification. Since training a CNN with an imbalanced dataset tends to yield a learning bias toward the major classes and eventually leads to deterioration in the classification accuracy, it is required to increase the number of training images for the minority classes to have evenly distributed training images over all classes. However, it is difficult to find the images with a similar facial emotion for the oversampling. In this paper, we propose to use the AU features to retrieve an image with a similar emotion. The query selection from the minority class and the AU-based retrieval processes repeat until the numbers of training data over all classes are balanced. Also, to improve the classification accuracy, the AU features are fused with the CNN features to train a support vector machine (SVM) for final classification. The experiments have been conducted on three imbalanced facial image datasets, RAF-DB, FER2013, and ExpW. The results demonstrate that the CNNs trained with the AU features improve the classification accuracy by 3%-4%.

Labeled images play an important role for training convolutional neural networks (CNN). In particular, training CNNs for facial emotion classification, the publicly available datasets suffer from noisy labels and inter-class imbalance problem. In this paper, we adopt a Generative Adversarial Network (GAN) to alleviate both noisy labeling and inter-class imbalance problems. Specifically, the noisy labelled images are identified by cross-checking the classified results with two fine-tuned CNNs and their facial emotions are strengthened by a GAN. Also, some of the neutral emotion images are transformed into minor emotion classes to solve the imbalance problem.

Conventional image resizing problems demand hard conditions on size and aspect ratio, which must be met with no tolerance. In this paper, a generalized optimization framework is presented, which can handle soft conditions as well as the hard ones. The soft condition can be given by an allowable range of the image parameter, which is incorporated as an inequality condition in the constrained optimization framework. Given the soft constraints, the proposed framework seeks to find the set of image parameters that minimize the cost function. A constrained optimization via a linear programming framework is employed to manage a diverse combination of soft and hard conditions for the target image. The optimization is based on the image line, which optimally selects a set of image lines (columns and rows) to be deleted for size reduction in accordance with the cost function and the constraints. As a case study, the line-based optimal image resizing method based on the linear programming framework is applied for the pre-processing of VGG-19 convolutional neural network (CNN). Although the target input size is a hard condition of $224\times 224$ for the VGG-19 CNN, the proposed optimization framework with a soft condition on the image size firstly finds an optimal near-square image with a tradeoff against the saliency level of image features. Then, the optimal near-square image is linearly scaled to the final image size to meet the hard condition.

Many devices in cloud environments support different spatial resolutions, necessitating image resizing of the original image contents. The goal of this paper is to combine multiple operators for image resizing in a stochastic optimization framework, seeking an optimal balance among essential resizing operators such as cropping and linear scaling. Specifically, we formulate image resizing as a MAP (maximum a posteriori) optimization problem with a Gibbs energy function. To reduce computational complexity we seek a sub-optimal solution of the MAP criterion with a deterministic implementation of the Metropolis algorithm. Since the optimization is carried out on the basis of a straight horizontal or vertical line in an image instead of the curved seam pixels, the optimization should converge quickly to have a fast image resizing. In addition, our image resizing can be associated with various user-defined content filtering such as a color masking. Finally, our resizing method is reversible, meaning that the image with the original size can be reconstructed from the retargeted image. This allows us to apply the proposed image resizing method to a prioritized image transmission with a scalable and progressive structure.

Vision-based detection of illegal or accidental activities in urban traffic has attracted great interest. Since state-of-the-art online automated detection algorithms are far from perfect, much research effort on offline video surveillance has been made to prevent police or security staff from observing all recorded video frames unnecessarily. To solve the problem, this study focuses on video condensation, which provides fast monitoring of moving objects in a long duration of surveillance videos. Considering the computational complexity and the condensation ratio as the two main criteria for efficient video condensation, we propose a video condensation algorithm, which consists of the following: 1) initial condensation by discarding frames of nonmoving objects; 2) intra-GoFM (group of frames with moving objects) condensation; and 3) inter-GoFM condensation. In the intra-GoFM and inter-GoFM condensation, spatiotemporal static pixels within each GoFM and temporal static pixels between two consecutive GoFMs are dropped to shorten the temporal distances between consecutive moving objects. Experimental results show that our video condensation saves a significant amount of computational loads compared with the previous methods without sacrificing the condensation ratio and visual quality.

Kinect v2 adopts a time-of-flight (ToF) depth sensing mechanism, which causes different type of depth artifacts comparing to the original Kinect v1. The goal of this paper is to propose a depth completion method, which is designed especially for the Kinect v2 depth artifacts. Observing the specific types of depth errors in the Kinect v2 such as thin hole-lines along the object boundaries and the new type of holes in the image corners, in this paper, we exploit the position information of the color edges extracted from the Kinect v2 sensor to guide the accurate hole-filling around the object boundaries. Since our approach requires a precise registration between color and depth images, we also introduce the transformation matrix which yields point-to-point correspondence with a pixel-accuracy. Experimental results demonstrate the effectiveness of the proposed depth image completion algorithm for the Kinect v2 in terms of completion accuracy and execution time.

Typical depth quality metrics require the ground truth depth image or stereoscopic color image pair, which are not always available in many practical applications. In this paper, we propose a new depth image quality metric which demands only a single pair of color and depth images. Our observations reveal that the depth distortion is strongly related to the local image characteristics, which in turn leads us to formulate a new distortion assessment method for the edge and non-edge pixels in the depth image. The local depth distortion is adaptively weighted using the Gabor filtered color image and added up to the global depth image quality metric. The experimental results show that the proposed metric closely approximates the depth quality metrics that use the ground truth depth or stereo color image pair.

In this paper, we propose a novel interpolation method to expand the decimated stereoscopic 3D (S3D) video to the original size. The basic approach for our interpolation is to exploit key-point correspondences between the stereoscopic left and right images. Since the rectified left and right frames of the S3D videos are aligned, a simple key-point detection method can be employed without considering the scale and transformation invariances. After detecting matched key-point pairs between the left and right images, we can interpolate the decimated pixels by exploiting the corresponding key-points in the opposite view as well as their neighboring pixels in the current view. The merit of our method is that no side information overhead is required for the interpolation. Nevertheless, the proposed method yields similar or even better PSNR performance than the previous side information method.

Human action recognition from a video scene has remained a challenging problem in the area of computer vision and pattern recognition. The development of the low-cost RGB depth camera (RGB-D) allows new opportunities to solve the problem of human action recognition. In this paper, we present a comprehensive review of recent approaches to human action recognition based on depth maps, skeleton joints, and other hybrid approaches. In particular, we focus on the advantages and limitations of the existing approaches and on future directions.

Detecting obstacles on the road is crucial for the advanced driver assistance systems. Obstacle detection on the road can be greatly facilitated if we have a vertical road profile. Therefore, in this paper, we present a novel method that can estimate an accurate vertical road profile of the scene from the stereo images. Unlike conventional stereo-based road profile estimation methods that heavily rely on a parametric model of the road surface, our method can obtain a road profile for an arbitrary complicated road. To this end, an energy function that includes the stereo matching fidelity and spatio-temporal smoothness of the road profile is presented, and thus the road profile is extracted by maximizing the energy function via dynamic programming. The experimental results demonstrate the effectiveness of the proposed method.

Depth maps taken by the low cost Kinect sensor are often noisy and incomplete. Thus, post-processing for obtaining reliable depth maps is necessary for advanced image and video applications such as object recognition and multi-view rendering. In this paper, we propose adaptive directional filters that fill the holes and suppress the noise in depth maps. Specifically, novel filters whose window shapes are adaptively adjusted based on the edge direction of the color image are presented. Experimental results show that our method yields higher quality filtered depth maps than other existing methods, especially at the edge boundaries.

The important requirements for stereo video retargeting are threefold: keeping temporal coherence, preventing depth distortion, and minimizing shape distortions of the retargeted video. To meet these requirements, the left and right video sequences are divided into groups of frames (GoFs), where the GoF is a basic unit for the seam carving and we assign a set of fixed seams for all frames within the GoF. To determine the fixed seams for each GoF, we need to find the GoF boundary in the video first. Then, the representative frame for each GoF is generated by considering the spatial saliency and temporal coherence. Also, the confidence of the stereoscopic correspondence between the left and right frames is considered to prevent depth distortion.

Kinect v2 provides depth images with reduced noises and holes, which simplifies the pre-processing of eliminating noises and filling holes. This also allows us to detect the moving object with the depth information of Kinect v2 without resorting to RGB image. Based on this a fast moving object detection method for Kinect v2 is presented in this paper. Our method performs very quickly while extracting moving objects accurately.

Haze removal, which is also referred to as image dehazing, has been extensively used to improve the visibility in images captured under inclement weather. In particular, the dark channel prior (DCP)-based single image dehazing has received the greatest amount of interest due to its superior performance. However, since the DCP is based on the characteristics of natural outdoor images, its reliability tends to decrease especially when an image contains man-made textures. In this paper, we present a DCP-based single image dehazing method that is robust when text or text-like patterns are present in the image. The proposed method first estimates the text likelihood from a hazy image using the stroke width transform (SWT) and uses the estimated likelihood to correct the DCP. The experimental results show that the proposed algorithm outperforms the conventional DCP-based dehazing methods.

The presence of haze in the atmosphere degrades the quality of images captured by visible camera sensors. The removal of haze, called dehazing, is typically performed under the physical degradation model, which necessitates a solution of an ill-posed inverse problem. To relieve the difficulty of the inverse problem, a novel prior called dark channel prior (DCP) was recently proposed and has received a great deal of attention. The DCP is derived from the characteristic of natural outdoor images that the intensity value of at least one color channel within a local window is close to zero. Based on the DCP, the dehazing is accomplished through four major steps: atmospheric light estimation, transmission map estimation, transmission map refinement, and image reconstruction. This four-step dehazing process makes it possible to provide a step-by-step approach to the complex solution of the ill-posed inverse problem. This also enables us to shed light on the systematic contributions of recent researches related to the DCP for each step of the dehazing process. Our detailed survey and experimental analysis on DCP-based methods will help readers understand the effectiveness of the individual step of the dehazing process and will facilitate development of advanced dehazing algorithms.

As a transition stage from a conventional 2D TV to a full stereoscopic 3D TV system, a frame-compatible format of fitting stereoscopic left and right images to a single frame of the existing 2D TV is required to utilize existing codec and transmission infrastructure. To meet this requirement, a frame-compatible top-bottom packing with a horizontal line offset is proposed, where the vertical resolutions of the stereoscopic left and right images are reduced by half. Then, the optimal interpolation mode for each line segment of the sub-sampled horizontal line is determined by exploiting parallax-compensated data as well as undeleted neighboring upper and lower horizontal lines. At the receiver, the discarded horizontal lines for the left and right images are reconstructed by the interpolation modes provided by the sender. Experimental results show that the proposed algorithm improves the PSNR as much as 1.5---3dB comparing to conventional interpolation filters.

Many researchers have studied iris recognition techniques in unconstrained environments, where the probability of acquiring non-ideal iris images is very high due to off-angles, noise, blurring and occlusion by eyelashes, eyelids, glasses, and hair. Although there have been many iris segmentation methods, most focus primarily on the accurate detection with iris images which are captured in a closely controlled environment. This paper proposes a new iris segmentation method that can be used to accurately extract iris regions from non-ideal quality iris images. This research has following three novelties compared to previous works; firstly, the proposed method uses AdaBoost eye detection in order to compensate for the iris detection error caused by the two circular edge detection operations; secondly, it uses a color segmentation technique for detecting obstructions by the ghosting effects of visible light; and thirdly, if there is no extracted corneal specular reflection in the detected pupil and iris regions, the captured iris image is determined as a ''closed eye'' image. The proposed method has been tested using the UBIRIS.v2 database via NICE.I (Noisy Iris Challenge Evaluation - Part I) contest. The results show that FP (False Positive) error rate and FN (False Negative) error rate are 1.2% and 27.6%, respectively, from NICE.I report (the 5th highest rank).

Evaluations on the quality of liquid crystal display (LCD) reveal that human factors yield more accurate results than conventional measurements such as image resolution and response speed. In this paper, as a measurement for the human factor, eyestrains on LCD TV are evaluated and analyzed in terms of video features such as edge difference and scene change. Our study is novel in the following three aspects. First, two measures such as the average blinking rate and the average pupil accommodation speed provide more reliable measurements on eyestrains. Second, video based features such as edge difference and scene change are used to examine the relation between eyestrain and video on LCD TV. Third, the guidelines for LCD TV manufacturer in terms of minimizing eyestrain are suggested. Experimental results show that more edge differences and scene changes induce the decrement of eyestrain. Thus, to boost edge difference and scene change, the amplification method of high frequency component on successive images in uncompressed or compressed domain is presented in order to manufacture the LCD TV with low eyestrains.

H.264/AVC for DMB is the most advanced video compression standard which provides a high compression rate and a video quality by adopting new technologies. However, these new technologies prevent us from applying some conventional algorithms directly to the H.264/AVC compression domain. For example, we need to study new image resizing schemes in the compressed bit-stream. So, in this paper, we propose a new thumbnail extraction method which can be applied to H.264. The proposed method shows that it can extract a 1/16 size thumbnail video, saving the operation time up to 50% to 70%.

Image segmentation is an important tool in image processing and can serve as an efficient front end to sophisticated algorithms and thereby simplify subsequent processing. We develop a multiclass image segmentation method using hidden Markov Gauss mixture models (HMGMMs) and provide examples of segmentation of aerial images and textures. HMGMMs incorporate supervised learning, fitting the observation probability distribution given each class by a Gauss mixture estimated using vector quantization with a minimum discrimination information (MDI) distortion. We formulate the image segmentation problem using a maximum a posteriori criteria and find the hidden states that maximize the posterior density given the observation. We estimate both the hidden Markov parameter and hidden states using a stochastic expectation-maximization algorithm. Our results demonstrate that HMGMM provides better classification in terms of Bayes risk and spatial homogeneity of the classified objects than do several popular methods, including classification and regression trees, learning vector quantization, causal hidden Markov models (HMMs), and multiresolution HMMs. The computational load of HMGMM is similar to that of the causal HMM.

Kinect v2, a new version of Kinect sensor, provides RGB, IR (Infrared) and depth images like its predecessor Kinect v1. However, the depth measurement mechanism and the image resolutions of the Kinect v2 are different from those of Kinect v1, which requires a new transformation matrix for the camera calibration of Kinect v2. In this paper, we correct the radial distortion of the RGB camera and find the transformation matrix for the correspondence between the RGB and depth image of the Kinect v2. Experimental results show that our method yields accurate correspondence between the RGB and depth images.

Different similarity measures between the descriptors of the key-points certainly yield different performances in image matching. In this paper we introduce an effective similarity measurement, which considers the distances of each key-point in a query image and its matched key-point with the smallest distance in the test image. Therefore, the distances of all key-points in the query image to the corresponding matched key-points in the test image contribute to the final similarity measurement. On the other hand, the previous method considers only the distances less than a threshold value of all possible key-point pairs, which may ignore a significant part of the key-points in the query image. Our experiments show that the proposed measure yields better performance for image similarity matching and retrieval.

In this paper, a new quality metric for depth images is proposed. Unlike the conventional depth metrics which require the additional information such as the ground truth depth image or a stereo image pair, the proposed quality metric demands only a single camera image and its corresponding depth image. In this work, we first empirically observe that the depth distortion is closely related to the local image characteristics. Based on the observation, we introduce a method to assess the local depth distortion for the edge and non-edge regions. Then, the local distortion is adaptively weighted by the Gabor filter and added up to the quality metric for the depth image.

Image retrieval based on color histograms requires quantization of a color space. Uniform scalar quantization of each color channel is a popular method for the reduction of histogram dimensionality. With this method, however, no spatial information among pixels is considered in constructing the histograms. Vector quantization (VQ) provides a simple and effective means for exploiting spatial information by clustering groups of pixels. We propose the use of Gauss mixture vector quantization (GMVQ) as a quantization method for color histogram generation. GMVQ is known to be robust for quantizer mismatch, which motivates its use in making color histograms for both the query image and the images in the database. Results show that the histograms made by GMVQ with a penalized log-likelihood (LL) distortion yield better retrieval performance for color images than the conventional methods of uniform quantization and VQ with squared error distortion.

MPEG-7 Visual Standard specifies a set of descriptors that can be used to measure similarity in images or video. Among them, the Edge Histogram Descriptor describes edge distribution with a histogram based on local edge distribution in an image. Since the Edge Histogram Descriptor recommended for the MPEG-7 standard represents only local edge distribution in the image, the matching performance for image retrieval may not be satisfactory. This paper proposes the use of global and semi-local edge histograms generated directly from the local histogram bins to increase the matching performance. Then, the global, semi-global, and local histograms of images are combined to measure the image similarity and are compared with the MPEG-7 descriptor of the local-only histogram. Since we exploit the absolute location of the edge in the image as well as its global composition, the proposed matching method can retrieve semantically similar images. Experiments on MPEG-7 test images show that the proposed method yields better retrieval performance by an amount of 0.04 in ANMRR, which shows a significant difference in visual inspection.

Edge histogram descriptor (EHD) consists of histogram bins of local edges in an image. Instead of the conventional bin-to-bin matching for the similarity measure, this paper presents a binary string descriptor for the EHD to achieve a faster binary bit-to-bit matching.

representative frame in GoF (Group of frames) of a video is formed by taking spatial and temporal gradients sequentially for image frames and by selecting the pixel of the largest spatial-temporal gradient (STG) for all co-located pixels in the GoF. As a result, the boundary of the moving object in the video is highlighted by the STG operation. Therefore, an optimal bounding box for a moving object can be determined by choosing the maximum spatial density of the STG for various sizes of the bounding box. The bounding box includes the boundary trace of the MOOI (moving object of interest) in the GoF and is used to differentiate the MOOI from the non-MOOI. That is, the pixels outside the bounding box are the non-MOOI and they are the main target for the size reduction of the video frames for a pre-processing of video compression.

A texture description is useful in many applications including similarity based image search and browsing. We present here three texture descriptors that are being considered for the final committee draft of the ISO/MPEG-7 standard. A comprehensive overview of the syntax and semantics of these texture descriptors is provided. The Homogeneous Texture Descriptor (HTD) and the Edge Histogram Descriptor (EHD) are useful in similarity search. The HTD characterizes homogeneous texture regions and is also useful in texture classification and recognition. The EHD is applicable when the underlying texture is not homogeneous and can also be used in sketch based retrieval. In addition, a compact descriptor that facilitates browsing applications is also defined. These descriptors are selected after a highly competitive test and evaluation phase within the MPEG group and we briefly summarize the evaluation criteria, the datasets used and the experimental results.

A novel function for image watermarking is proposed. We show how the watermarking sequence can be used for correction of illegal modifications. To make this task possible, the parities generated by the conventional error control coding (ECC) technique is used for the watermarking sequence. Then, the receiver can correct any alterations by applying the ECC decoding. To increase the correction capability we also adopt a scrambling scheme before the encoding to change a burst type of content modifications (i.e., errors) into a random noise. The scrambling key can be also used as a key for the authentication of the sender.

An image fidelity assessment using the edge histogram descriptor (EHD) of MPEG-7 is presented. Neither additional data nor fragile watermarking is needed, and there is no need to access the original image as a reference. Only the EHDs of the original image and the received image are required. The peak signal-to-noise ratio (PSNR) obtained by comparing the EHD extracted from the received image and that of the original image is used to assess the noise level of the received image. Experimental results show that the PSNRs calculated from the conventional pixel-to-pixel gray level and from the proposed bin-to-bin EHD maintain a proportional relationship. This implies that the EHD can be used instead of image data for the image fidelity assessments.

Query-by-example (QBE) is a well-known method for image retrieval. In reality, however, an example image to be used for the query is rarely available. Therefore, it is often necessary to find a good example image to be used for the query before applying the QBE method. Query-by-layout (QBL) is our proposal for that purpose. In particular, we make use of the visual descriptors such as the edge histogram descriptor (EHD) and the color layout descriptor (CLD) in MPEG-7. Since image features of the CLD and the EHD can be localized in terms of a 4 4 sub-image, we can specify image features such as color and edge distribution on each sub-image separately for image retrieval without a query image. Experimental results show that the proposed query method can be used to retrieve a good image as a starting point for further QBE-based image retrieval.

A novel block-based image segmentation algorithm using the maximum a posteriori (MAP) criterion is proposed. The conditional probability in the MAP criterion, which is formulated by the Bayesian framework, is in charge of classifying image blocks into edge, monotone, and textured blocks. On the other hand, the a priori probability is responsible for edge connectivity and homogeneous region continuity. After a few iterations to achieve a deterministic MAP optimization, we can obtain a block-based segmented image in terms of edge, monotone, or textured blocks. Then, using a connected block-labeling algorithm, we can assign a number to all connected homogeneous blocks to define an interior of a region. Finally, uncertainty blocks, which are not given any region number yet, are assigned to one of the neighboring homogeneous regions by a block-based region-growing method. During this process, we can also check the balance between the accuracy and the cost of the contour coding by adjusting the size of the uncertainty blocks. Experimental results show that the proposed algorithm yields larger homogeneous regions which are suitable for the object-based image compression.

Frequency features of stereo images are investigated in the DFT (Discrete Fourier Transform) domain by characterizing phase and magnitude properties originated from the horizontal disparities in the stereo images. Also, the well-known DFT properties including the conjugate symmetry property are utilized to identify essential frequency components of stereo images. Our investigation reveals that the DFT of the stereo images has useful properties that can prioritize the DFT coefficients for compact representations and compressions.

Depth maps taken from depth cameras are often available at a lower resolution compared to the corresponding color images. In this paper, we propose a selective interpolation method to expand depth image to the size of the corresponding color image. To achieve our goal we selectively adopt either the bilinear smoothing filter or an edge-preserving filter to expand the low resolution depth map. The edge-preserving filters we considered in this paper are joint bilateral up-sampling (JBU) filter and the New Edge-Directed Interpolation (NEDI) filter. Our method not only maintains the visual quality of the interpolated depth image at edge regions, but it also reduces the computational complexity for real time execution.

In this paper, two considerations were presented for designing pico-projector based-on LCoS panel and LED. To have a clear picture, LED duty cycle has to be less than 50% of whole possible LED illumination times. And, a constant current circuit is needed to maintain a white balance of picture.

Stereoscopic 3D video is decimated by frame-compatible top-bottom packing structure. At the receiver the discarded horizontal lines are reconstructed by interpolation functions of parallax-compensated data as well as remained neighboring lines.

A clustering criterion introduced by Symons (1981), which is called Classification Maximum Likelihood (CML) criterion in this paper, is designed to consider the cluster size and the covariance structure of samples. The CML criterion is optimized by the ‘Moving method’ suggested by Duda and Hart (1973, p. 226). When the Moving method is applied to the CML criterion with an arbitrary initial cluster, it often yields degenerate clusters. To avoid such degenerate cases, we propose two stages of clustering. In the first stage, we roughly partition samples with respect to ‘the covariance structure component’ in the CML criterion. The resulting partition is then further clustered with the full CML criterion.

Differentiating region-of-interest (ROI) from non-ROI in an image in terms of relative size as well as fidelity becomes an important functionality for future visual communication environment with a variety of display devices. In this paper, we propose a scalable image representation with the ROI functionality in the spatial domain, which allows us to generate a hierarchy of images with arbitrary sizes. The ROI functionality of our scalable representation is a result of a nonuniform grid transformation in the spatial domain, where only the center of ROI and an expansion parameter are to be known. Our grid transformation guarantees no loss of information within the area of ROI.

This paper proposes a general unsupervised segmentation algorithm which estimates all model parameters, including the number of regions, as part of the segmentation. The general image model is a hierarchical one, consisting of Markov random fields as components of the model. The MAP criterion is adopted, in principle, for the simultaneous image segmentation and parameter estimation procedures. Due to the difficulty of implementing the MAP segmentation with a large number of unknown parameters, a novel modification of the MAP criterion is proposed. For the model parameters having closed-form ML estimates, these estimates are substituted back into the objective function to reduce the difficulty of the maximization. The remaining maximization is implemented by a recursive segmentation-parameter estimation algorithm, which yields a partial optimal solution (POS) to the maximization problem. In the special case where all model parameters have closed-form ML estimates, the proposed algorithm is equivalent to implementing the MAP criterion. The number of regions in the image is determined through a model fitting criterion tagged on to the segmentation algorithm. Special forms of the general unsupervised segmentation algorithm are developed for the segmentation of noisy and textured images. For noisy images, the image is assumed to consist of uniform graylevel regions modeled by a class of Gibbs random fields and corrupted by additive, white, region-dependent, Gaussian noise. For textured images, the image is assumed to consist of regions, modeled by a class of Gibbs random fields, which are filled with textures, modeled by Gaussian Markov random fields. The algorithms for both classes of images are applied to a wide range of images—generated according to the model, hand-drawn, natural and Brodatz textures, their combinations, and outdoor images—with notable success. Despite the large number of unknown parameters (as many as 14 for some noisy images and 36 for some textured images), the algorithms yield good segmentations, accurate estimates for the parameters, and the correct number of regions.

The authors propose two decoding strategies for Reed-Solomon product codes, and analyze their performance on a random error channel. The performance measures of interest are the probability of undetected error at the inner decoder output and that of decoding failure at the outer decoder output. It has been shown that the proposed decoding strategies give a significant performance improvement over the conventional decoder. >

To automatically generate images with the same foreground but different backgrounds, a watermark bit (e.g., binary 1 for foreground and 0 for background) can be inserted for each pixel location. Then, the embedded watermark bit can be automatically extracted and the background can be separated from the object. Note that the object extraction can be done successfully only if the watermarked image is intact. However, if the watermarked image goes through some post-processing including JPEG compression and cropping, then the pixel-wise watermark decoding may fail. To overcome this problem, in this paper, a block-wise watermark insertion and a block-wise MAP (maximum a posteriori) watermark decoding are proposed. Experimental results show that the proposed method is more robust that the pixel-wise decoding for various post-processing attacks.

To reduce the semantic gap, image retrieval systems based on users' relevance feedback have been adopted. However, since this structure needs human intervention during the retrieval process, it cannot be applied to fully automated systems. To avoid this problem, we propose a feed-forward framework instead of the feed-back retrieval system, which adds a classifier to the traditional system for giving feed-forward information to maximize the average precision. That is, given a database, our proposed system improves the overall precision by selecting the best mode based on known statistics (average precision vs. recall for each category). Lloyd-clustered Gauss mixtures are used in the classifier to provide the feed-forward category information and in the quantization of color images for histogram generation.

Object segmentation and extraction play an important role in computer vision and recognition problems. Unfortunately, with current computing technologies, fully automatic object segmentation is not possible, but human intervention is needed for outlining the rough boundary of the object to be segmented. The goal of this paper is to make the object extraction automatic after the first semi-automatic segmentation. That is, once the semantically meaningful object such as a house or a human body is extracted from the image under human's guidance, an image manipulation technique is applied. There is no noticeable difference between the original and the manipulated images. However, the embedded signature by the image manipulation can be detected automatically to be used to differentiate the object from the background. The manipulated images, which is called automatic-object-extractible images, can be used to provide training images with the same object but various background images.

An important issue in embedding watermark bits in compressed video stream is to keep the bit-rate unchanged after the watermarking. This is a very difficult problem for high efficient compression methods such as H.264/AVC, because just one bit alteration in highly compressed bit-stream may widely affect the video content. In this paper we solve this problem by embedding watermark bit to the sign bit of the Trailing Ones in Context Adaptive Variable Length Coding (CAVLC) of H.264/AVC. The algorithm yields no bit-rate change after the data hiding. Also, we can easily balance between the capacity of the watermark bits and the fidelity of the video. The simplicity of the proposed algorithm is an added bonus for the real-time applications. Our experiments show that the PSNRs of the video sequences after the data hiding are higher than 43dB.

A novel image quality assessment using the edge histogram descriptor (EHD) of MPEG-7 is presented. Neither additional data nor fragile watermarking is needed for the quality assessment and the image content authentication. Also, the original image is not needed for our method, no need to access the original image as a reference. Only the EHD metadata of the original image and the received (noisy or altered) one are required. The PSNR (Peak to Signal-to-Noise Ratio) or the mean-square error (MSE) is obtained by comparing the EHD extracted from the received image and that of the original image attached as the meta-data. Then, it is used to assess the level of the image degradation and any illicit modification of the image. Experimental results show that the PSNRs calculated from the two EHDs are similar to those calculated from the pixel-to-pixel comparisons of original and received images. This implies that one can use the EHD, instead of the image data, to calculate the PSNR for the image assessment. Also, since the EHD extracted from the received image is prone to be changed according to the alterations of the image content, one can also use the proposed method as the image authentication purpose.