Search

Image denoising methods generally remove not only noise but also fine-scale textures and thus degrade the subjective image quality. In this paper, we propose a method of recovering the texture component that is lost under a state-of-the-art denoising method called weighted nuclear norm minimization (WNNM). We recover the image texture with a linear minimum mean squared error estimator (LMMSE filter), which requires statistical information about the texture and noise. This requirement is the key problem preventing the application of the LMMSE filter for texture recovery because such information is not easily obtained. We propose a new method of estimating the necessary statistical information using Stein’s lemma and several assumptions and show that our estimated information is more accurate than the simple estimation in terms of the Fréchet distance. Experimental results show that our proposed method can improve the objective quality of denoised images. Moreover, we show that our proposed method can also improve the subjective quality when an additional parameter is chosen for the texture to be added.

Non-local self-similarity (NLSS) is widely used as prior information in an image restoration method. In particular, a low-rankness-based prior has a significant effect on performance. On the other hand, a number of color extensions of NLSS-based grayscale image restoration methods have been developed. These extensions focus on the pixel-wise correlation among color channels. However, a natural color image also has a complex dependency, known as an inter-channel dependency, among local regions from different color channels. As a result, color artifacts appear in a denoised image obtained by using the existing methods. In this paper, we propose a novel non-local and inter-channel dependency-aware prior called the weighted tensor nuclear norm (WTNN). The proposed prior is derived by incorporating inter-channel dependency to low-rank-based NLSS prior. The WTNN is a low-rankness-of-the-third-order patch tensor, and we apply it to the tensors constructed with non-local similar patches. It enables us to naturally represent the higher-order dependencies among similar color patches. We propose an image denoising algorithm using the WTNN and image restoration algorithm by using a non-trivial generalization of this algorithm. The experimental results clearly show that the proposed WTNN-based color image denoising and restoration algorithms outperform state-of-the-art methods.

Convolutional neural network (CNN)-based image denoising methods typically estimate the noise component contained in a noisy input image and restore a clean image by subtracting the estimated noise from the input. However, previous denoising methods tend to remove high-frequency information (e.g., textures) from the input. It caused by intermediate feature maps of CNN contains texture information. A straightforward approach to this problem is stacking numerous layers, which leads to a high computational cost. To achieve high performance and computational efficiency, we propose a gated texture CNN (GTCNN), which is designed to carefully exclude the texture information from each intermediate feature map of the CNN by incorporating gating mechanisms. Our GTCNN achieves state-of-the-art performance with 4.8 times fewer parameters than previous state-of-the-art methods. Furthermore, the GTCNN allows us to interactively control the texture strength in the output image without any additional modules, training, or computational costs.

Summary Ground-penetrating radar (GPR) is a geophysical method for non-destructive inspection of underground infrastructure. The main impediment to machine-learning-based classification of GPR data is gathering enough labeled data. Previous work done to solve this problem generated pseudo-GPR data through the numerical simulations done with expensive GPU clusters. In this paper, we propose a simple yet effective framework for machine learning with GPR data without enormous computational cost. The key idea of our method is quasi-measurement pitch changes (QMPC) that can obtain several times the amount of pseudo-data from real measurements. QMPCs are based on a simple sub-sampling procedure from real data, and no interpolation is applied for the pseudo-data. Thus, special hardware like GPU clusters are not required and no artifacts are produced by such an interpolation. Moreover, using QMPCs for test data allows us to apply ensemble learning at the inference phase of machine learning. The experimental results for the classification problem of buried objects clearly show that our framework can drastically improve accuracy with additional labeled data and without significantly increasing computational cost.

Although non-local image denoising has attracted much research effort due to its superior performance, little attention has focused on its color extension. Most existing non-local color image denoising methods process the color channels of an input image separately. However, in order to improve the performance of color image denoising, all color channels should be processed jointly for fully utilizing the interchannel dependency. This paper proposes a new non-local and inter-channel dependency aware prior, named weighted tensor nuclear norm (WTNN), and it is defined on a 3rd-order tensor from a patch cluster of an input image. We also present an effective algorithm for color image denoising using the WTNN. Experimental results clearly show that the proposed algorithm outperforms a state-of-the-art color image denoising method, known as CBM3D.

This paper proposes a method to recover a texture component which has been lost by weighted nuclear norm minimization – the current state of the art of image denoising. Based on a non-trivial assumption on a statistic between the texture and noise, we can estimate the statistic by using the Stein's lemma. It allows us to recover the texture effectively by using a linear minimum mean squared error estimator (Wiener filter). The experimental results show that our proposed method can improve the image recovery performance of weighted nuclear norm minimization for image denoising (WNNM) in both quantitative and qualitative evaluation.

We propose a new color imaging system based on a compressive sensing technique. Our system consists of a random complementary color filter array (CFA) for random projection and a color reconstruction method for demosaicing. Our CFA overlaps two complementary color filters and consists of six color filters: cyan (C), yellow (Y), magenta (M), C+Y, C+M, and Y+M. By arranging these six color filters randomly, our imaging system achieves pseudo random projection among red (R) /green (G) / blue (B) colors, which is the key technology of compressive sensing. Because this CFA can retain more color information than RGB CFA, the proposed color reconstruction method reduces artifacts at monochromatic edges and in high-frequency regions, and obtains better image quality. As an additional contribution, we introduce saturation consistency to suppress color artifacts in saturated areas, then achieve to 3.3 dB higher quality images than the conventional method.

Recently, a novel approach to color image compression based on colorization has been presented. The conventional method for colorization-based image coding tends to lose the local oscillation of chrominance components that the original images had. A large number of color assignments is required to restore these oscillations. On the other hand, previous studies suggest that an oscillation of a chrominance component correlates with the oscillation of a corresponding luminance component. In this paper, we propose a new colorization-based image coding method that utilizes the local correlation between texture components of luminance and chrominance. These texture components are obtained by a total variation regularized energy minimization method. The local correlation relationships are approximated by linear functions, and their coefficients are extracted by an optimization method. This key idea enables us to represent the oscillations of chrominance components by using only a few pieces of information. Experimental results showed that our method can restore the local oscillation and code images more efficiently than the conventional method, JPEG, or JPEG2000 at a high compression rate.

Many content distribution systems such as CDN and P2P file sharing have been developed. In these systems, file-type contents require downloads to be completed before they can be played and they have no value before the download finishes. Therefore, a user's satisfaction depends on the length of the service latency. That is, the length of time from when the user issued a request until the user received an entire file. Reducing the sum of that time is necessary for the whole delivery system to satisfy users and maintain dependability on system performance. We discuss a hop-by-hop file delivery system suitable for delivering file contents whereby the sum of service latency is reduced by using the request conditions. Moreover, we propose a file delivery scheduling algorithm for a one-link model given that the content request frequency is unknown. The algorithm is based on a local optimal strategy. We performed a characteristic analysis by computer simulation. The results showed that our algorithm performs at nearly the theoretical efficiency limit of the hop-by-hop system when the request frequency distribution of each content has a deviation.

Novel image compression approaches that use a process called colorization, which is a method for adding color to a monochrome image, has recently been proposed. Almost all the proposed methods represent color information by using a set of color points. However, in an area with flat chrominance and variant luminance, we need redundant points to correctly decode color. We propose a color information extraction method that represents color information as a set of color lines. To effectively extract lines from an input image, color clustering and tentative color information have been introduced. The experimental results show that our proposed method allows us to drastically suppress the color information compared to existing colorization-based coding and conventional JPEG compression.

Many content distribution systems such as CDN and P2P file sharing have been developed. In these systems, file-type contents require downloads to be completed before they can be played and they have no value before the download finishes. Therefore, the total download time for this sort of content is very important, and the service period should be reduced to satisfy users. We discuss a hop-by-hop file delivery system suitable for delivering file contents whereby the sum of service periods is reduced by using the request conditions. Moreover, we propose a file delivery scheduling algorithm for a one-link model given that the content request frequency is unknown. The algorithm is based on a local optimal strategy. We performed a characteristic analysis by computer simulation. The results showed that our algorithm performs to reduce the sum of the service periods in the hop-by-hop system effectively when request frequency distribution of each content has a deviation.

The regularization terms play an important role for solving various color image recovery problem via convex optimization algorithms. Recently, the decorrelated vectorial total variation (D-VTV) is proposed as such a regularizer. The D-VTV decorrelates RGB-color channels by using 3-point DCT. However, we reveal that some relations still exist among decorrelated color channels. In this paper, we proposed a new convex regularization term named inter channel relation based vectorial total variation (IC-VTV) and its higher order extension for utilizing the relationship between decorrelated color channels. Experimental results show that our proposed prior can outperform the conventional methods.

We propose a video coding method for achieving desired subjective quality. In general, subjective video quality is most strongly affected by the section with the lowest quality. Therefore, to efficiently achieve the desired subjective quality, it is necessary to minimize the variation in subjective quality over time. This approach requires variable-bit-rate, variable-frame-rate video coding and methods for estimative subjective quality. We previously proposed such methods for temporal quality (motion smoothness) and spatial quality (quality of each frame). Applying these methods enables us to develop a simple method for achieving the desired subjective quality. We implemented the proposed method on the general MPEG-4 encoder. Experimental results show that this method achieves the desired quality.

Using a novel characterization of texture, we propose an image decomposition technique that can effectively decomposes an image into its cartoon and texture components. The characterization rests on our observation that the texture component enjoys a blockwise low-rank nature with possible overlap and shear, because texture, in general, is globally dissimilar but locally well patterned. More specifically, one can observe that any local block of the texture component consists of only a few individual patterns. Based on this premise, we first introduce a new convex prior, named the block nuclear norm (BNN), leading to a suitable characterization of the texture component. We then formulate a cartoon-texture decomposition model as a convex optimization problem, where the simultaneous estimation of the cartoon and texture components from a given image or degraded observation is executed by minimizing the total variation and BNN. In addition, patterns of texture extending in different directions are extracted separately, which is a special feature of the proposed model and of benefit to texture analysis and other applications. Furthermore, the model can handle various types of degradation occurring in image processing, including blur+missing pixels with several types of noise. By rewriting the problem via variable splitting, the so-called alternating direction method of multipliers becomes applicable, resulting in an efficient algorithmic solution to the problem. Numerical examples illustrate that the proposed model is very selective to patterns of texture, which makes it produce better results than state-of-the-art decomposition models.

This paper proposes an objective measurement for coded video quality based on camera work and frame quality. The same technique used in the objective measurement for coded still image is used to evaluate the frame quality. The proposed measurement also automatically detects the camera work. Experimental results showed that the estimation accuracy for the proposed measurement is improved. In particular, in the case of a still scene, the accuracy is improved greatly.

Image retargeting is a technique for displaying an image on various devices adaptively. However, the conventional retargeting methods restrict target display's shape as rectangular. For expanding their application field, generalization of the shape of target display is useful. Yet, generalization of the existing methods is non-trivial due to the fact that the basic idea of their algorithm deeply rooted in this shape restriction. On the other hand, one of the retargeting approaches, so called warping based method that have high affinity with optimization problem, is proposed. In this paper, we propose a warping based generalized image retargeting. For generalizing the warping method, the main problem is to solve a repositioning problem of regions of interest (ROIs) in input image. By restricting their movable regions, we convexificate this problem and it allows us to use a conventional convex optimization algorithm. The obtained result shows that our method outperforms the conventional method.

In recent years, production technologies for arbitrary-shaped (non-rectangular shaped) displays to improve the appearance of digital photo frames or to increase the advertising effectiveness of digital signs are proposed. To utilize such arbitrary-shaped displays effectively, an image retargeting method is required for converting a rectangular shaped image into an arbitrary-shaped image while preserving the information of the input image. However, conventional image retargeting methods can only convert a rectangular shaped image into a rectangular shaped image with different size or aspect ratio. In this paper, we propose a new image retargeting method for arbitrary-shaped images by dividing the image retargeting problem into two phases: moving important regions and inpainting unimportant regions. In the first phase, we find several important regions from the (rectangular shaped) input image using a block matching procedure. Then, we move these regions to the watching region of the target (arbitrary-shaped) display by minimizing the proposed energy functional with a greedy algorithm. After moving these regions, we inpaint the rest of the image to preserve the consistency with the input image. The experimental results show that we can obtain natural and structure-preserved images with arbitrary-shape by our proposed method.

We propose a new semi-norm named luminance constrained total variation (LCTV) to utilize a correlation and an imbalance of information amounts between luminance and chrominance for solving image recovery problems. As an example of LCTV application, we also propose a method for restoring chrominance components of JPEG 2000 compressed images. We cast this method into an l 1 (LCTV)-l 2 combined optimization problem. The algorithm to solve this optimization problem by using variable splitting is also proposed and its convergence is confirmed. Experimental results show our proposed optimized decoding method outperforms the existing state-of-the-art denoising methods.

Solving image restoration problems requires the use of efficient regularization terms that represent certain features of the original image. Natural images generally have three features: smooth regions, textures, and edges. However, conventional optimization techniques typically adopt only one or two reg-ularization terms, and there is no regularized optimization problem that represents such features exactly and completely. By applying three regularization terms corresponding to these three features, we can restore images more efficiently in ill-posed conditions. We propose here total variation (TV), wavelet, and curvelet regularized optimization for image restoration. These regularization terms correspond exactly to the smooth region, textures, and edges. We also present an algorithm to solve the proposed optimization problem, and ensure its convergence. Experimental results revealed that our optimization technique was more effective for image restoration than conventional methods.

Networks including high-functional nodes (HNs) that have recovery processes are currently being studied to improve the quality of service of live streaming. To utilize the HN recovery processes effectively, Novel Link Weight Method has been proposed as a path configuration method for HN networks, but this method sometimes configures inappropriate paths because it does not consider the interactions of link delays for link weight. Thus, we propose a path configuration method that takes into account the interactions of delay and loss among all the links on a path. Our proposed method uses the probability distribution of the link delay and convolution of the probability distribution. In a simulation, we showed that our proposed method is more suitable for path configuration for HN networks than other conventional methods.

Colorization is a method that adds color components to a grayscale image using only a few representative pixels provided by the user. A novel approach to image compression called colorization-based coding has recently been proposed. It automatically extracts representative pixels from an original color image at an encoder and restores a full color image by using colorization at a decoder. However, previous studies on colorization-based coding extract redundant representative pixels and do not extract the pixels required for suppressing coding error. This paper focuses on the colorization basis that restricts the decoded color components. From this viewpoint, we propose a new colorization-based coding method. Experimental results revealed that our method can drastically suppress the information amount (number of representative pixels) compared conventional colorization based-coding while objective quality is maintained.

Recently, a novel approach to color image compression based on colorization has been presented. Although the conventional method of colorization-based coding outperforms JPEG in terms of subjective quality, the decoded chrominance components lose the local oscillation that the original images had. A large number of color assignations is required to restore these oscillations. We focus on the local correlation that exists between luminance and chrominance in separated texture components, and we present a new colorization-based coding method. Experimental results showed that our coding method can restore the oscillation and improve the coding efficiency compared with the conventional method.

Total variation regularization can be used to decompose any natural image into a structure image (edges and smooth areas) and a texture image (texture only). Although structure images can be highly compressed by conventional JPEG methods, texture images require many bits to represent their feature details. Texture images are composed of several repeated patterns, each of which can be synthesized from a small texture image, and the borders of different texture patterns correspond to the edges in a structure image. We present a novel image compression method for these images based on this observation. First, we apply a context-aware resizing method to the input image to obtain a compaction image that has as many unique texture patterns as possible. Then the compaction image is divided into texture and structure images. Our proposed encoder sends the compressed compaction texture and compaction structure images and the compressed structure image extracted from the input image to the decoder side. At the decoder side, a texture image is synthesized from the compaction texture image through matching between the compaction structure image and the original-size structure image. Experimental results show that the decoded image obtained by our proposed method is subjectively similar to the original one, with higher texture feature accuracy than that obtained by a conventional JPEG method but almost the same data size.

Total variation regularization can be used to decompose any natural image into a structure image (edges and smooth areas) and a texture image (texture only). Although structure images can be highly compressed by conventional JPEG methods, texture images require a lot of bits to represent their feature details. Texture images are composed from several repeated patterns, each of which can be synthesized from small texture patches, and we propose a novel image compression method for these images. First, we extract several texture patches from an original texture image. Next, we compress the texture distribution map to a small number of representative points corresponding with each texture patch. The texture image is then synthesized from the texture distribution map and texture patches at the decoder. Experimental results show that the decoded image obtained by our proposed method is subjectively similar to the original one compared with the decoded image by the conventional JPEG.

Flow convergence on particular nodes causes various problems such as degradation in QoS. We describe a new routing method for reducing the maximum number of relay flows on the node that relays maximum flows (called “the maximum number of relay flows”). A full search is required for calculating the optimal algorithm of this problem. Thus, we created an approximate algorithm to calculate a near optimal solution using scalable computational costs. Moreover, when this method permits a call loss, the maximum number of relay flows can be reduced more. However, there is generally a trade-off relationship between the call loss rate and the maximum number of relay flows. Thus, we show the numerical relationship between the call loss rate and the maximum number of relay flows. Finally, we show, using a simulation in various network topology and traffic situations, that the maximum number of relay flows and the network delay can be reduced by our proposed method.

It is important to use an ALM (application layer multicast) tree with broad bandwidth to ensure the quality of streaming applications. However, when constructing an ALM tree without information on underlay traffic, we cannot know how much congestion there is in links. Therefore, if unexpected congestion in links occurs, the quality of ALM transmissions may worsen. However, no polynomial-time algorithms for constructing maximum-bandwidth ALM trees with information on underlay traffic exist. We regarded an underlay network as a tree network topology in our previous work, and presented a polynomial-time algorithm that was used to construct a maximum-bandwidth ALM tree. We also proved that we could obtain an optimal ALM tree. However, we had not yet evaluated our algorithm by simulation. Here, we discuss our evaluation of the algorithm we propose and a conventional one. We found our algorithm could achieve a broader bandwidth ALM tree at lower computational cost, especially in large networks.

Colorization is the process that restores colors on a grayscale image from user-generated color-assignation information. A novel approach to image compression has recently been proposed that extracts such color assignation from an input color image (we called this inverse colorization). Previous studies on inverse colorization have represented color assignation as a set of color points. However, in regions with flat color and fluctuating luminance, numerous color points are needed to correctly resore the color. Therefore, we propose a novel method of inverse colorization that represents color assignation in a more human-like manner - as a set of color line segments. To extract the minimum necessary line segments from an input image, iterative updating of tentative color assignation was introduced. The experimental results revealed that our proposed method can drastically suppress color information compared to either conventional inverse colorization or JPEG.

To perform Quality of Service (QoS) control of video communication more efficiently, it is necessary to develop an objective quality evaluation method for coded video. Many proposed conventional methods to obtain video quality require the availability of both reference and processed video sequence. However, in case of re-encoding the coded video stream at the receiver side where reference video sequence is not present, it is impossible to do such a full-reference evaluation. Therefore, we have developed a video quality evaluation model by using reduced reference for evaluated value obtained by SSCQE method. In this approach, we use some features extracted from reference video. It is called reduced-reference method by VQEG. Transmitting these features with coded video, the proposed model can estimate the video quality, even in the absent of the full original video in the decoder side. The video quality rating obtained from the proposed model shows good agreement with subjective quality.© (2003) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.