Search

Location awareness is a critical issue in wireless sensor network applications. For more accurate location estimation, the two issues should be considered extensively: 1) how to sufficiently utilize the connection information between multiple nodes and 2) how to select a suitable solution from multiple solutions obtained by the Euclidean distance loss. In this paper, a DV-Hop localization based on the distance estimation using multinode (DEMN) and the hop loss in WSNs is proposed to address the two issues. In DEMN, when multiple anchor nodes can detect an unknown node, the distance expectation between the unknown node and an anchor node is calculated using the cross-domain information and is considered as the expected distance between them, which narrows the search space. When minimizing the traditional Euclidean distance loss, multiple solutions may exist. To select a suitable solution, the hop loss is proposed, which minimizes the difference between the real and its predicted hops. Finally, the Euclidean distance loss calculated by the DEMN and the hop loss are embedded into the multi-objective optimization algorithm. The experimental results show that the proposed method gains 86.11\% location accuracy in the randomly distributed network, which is 6.05% better than the DEM-DV-Hop, while DEMN and the hop loss can contribute 2.46% and 3.41%, respectively.

Existing image compressed sensing (CS) coding frameworks usually solve an inverse problem based on measurement coding and optimization-based image reconstruction, which still exist the following two challenges: 1) The widely used random sampling matrix, such as the Gaussian Random Matrix (GRM), usually leads to low measurement coding efficiency. 2) The optimization-based reconstruction methods generally maintain a much higher computational complexity. In this paper, we propose a new CNN based image CS coding framework using local structural sampling (dubbed CSCNet) that includes three functional modules: local structural sampling, measurement coding and Laplacian pyramid reconstruction. In the proposed framework, instead of GRM, a new local structural sampling matrix is first developed, which is able to enhance the correlation between the measurements through a local perceptual sampling strategy. Besides, the designed local structural sampling matrix can be jointly optimized with the other functional modules during training process. After sampling, the measurements with high correlations are produced, which are then coded into final bitstreams by the third-party image codec. At last, a Laplacian pyramid reconstruction network is proposed to efficiently recover the target image from the measurement domain to the image domain. Extensive experimental results demonstrate that the proposed scheme outperforms the existing state-of-the-art CS coding methods, while maintaining fast computational speed., Comment: Accepted by ACM Transactions on Multimedia Computing Communications and Applications (TOMM)

Localization is one of the pivotal issues in wireless sensor network applications. In 3D localization studies, most algorithms focus on enhancing the location prediction process, lacking theoretical derivation of the detection distance of an anchor node at the varying hops, engenders a localization performance bottleneck. To address this issue, we propose a probability-based average distance estimation (PADE) model that utilizes the probability distribution of node distances detected by an anchor node. The aim is to mathematically derive the average distances of nodes detected by an anchor node at different hops. First, we develop a probability-based maximum distance estimation (PMDE) model to calculate the upper bound of the distance detected by an anchor node. Then, we present the PADE model, which relies on the upper bound obtained of the distance by the PMDE model. Finally, the obtained average distance is used to construct a distance loss function, and it is embedded with the traditional distance loss function into a multi-objective genetic algorithm to predict the locations of unknown nodes. The experimental results demonstrate that the proposed method achieves state-of-the-art performance in random and multimodal distributed sensor networks. The average localization accuracy is improved by 3.49\%-12.66\% and 3.99%-22.34%, respectively.

Guided depth map super-resolution (GDSR), which aims to reconstruct a high-resolution (HR) depth map from a low-resolution (LR) observation with the help of a paired HR color image, is a longstanding and fundamental problem, it has attracted considerable attention from computer vision and image processing communities. A myriad of novel and effective approaches have been proposed recently, especially with powerful deep learning techniques. This survey is an effort to present a comprehensive survey of recent progress in GDSR. We start by summarizing the problem of GDSR and explaining why it is challenging. Next, we introduce some commonly used datasets and image quality assessment methods. In addition, we roughly classify existing GDSR methods into three categories, i.e., filtering-based methods, prior-based methods, and learning-based methods. In each category, we introduce the general description of the published algorithms and design principles, summarize the representative methods, and discuss their highlights and limitations. Moreover, the depth related applications are introduced. Furthermore, we conduct experiments to evaluate the performance of some representative methods based on unified experimental configurations, so as to offer a systematic and fair performance evaluation to readers. Finally, we conclude this survey with possible directions and open problems for further research. All the related materials can be found at \url{https://github.com/zhwzhong/Guided-Depth-Map-Super-resolution-A-Survey}., Comment: Accepted by ACM Computing Surveys

Existing industrial anomaly detection (IAD) methods predict anomaly scores for both anomaly detection and localization. However, they struggle to perform a multi-turn dialog and detailed descriptions for anomaly regions, e.g., color, shape, and categories of industrial anomalies. Recently, large multimodal (i.e., vision and language) models (LMMs) have shown eminent perception abilities on multiple vision tasks such as image captioning, visual understanding, visual reasoning, etc., making it a competitive potential choice for more comprehensible anomaly detection. However, the knowledge about anomaly detection is absent in existing general LMMs, while training a specific LMM for anomaly detection requires a tremendous amount of annotated data and massive computation resources. In this paper, we propose a novel large multi-modal model by applying vision experts for industrial anomaly detection (dubbed Myriad), which leads to definite anomaly detection and high-quality anomaly description. Specifically, we adopt MiniGPT-4 as the base LMM and design an Expert Perception module to embed the prior knowledge from vision experts as tokens which are intelligible to Large Language Models (LLMs). To compensate for the errors and confusions of vision experts, we introduce a domain adapter to bridge the visual representation gaps between generic and industrial images. Furthermore, we propose a Vision Expert Instructor, which enables the Q-Former to generate IAD domain vision-language tokens according to vision expert prior. Extensive experiments on MVTec-AD and VisA benchmarks demonstrate that our proposed method not only performs favorably against state-of-the-art methods under the 1-class and few-shot settings, but also provide definite anomaly prediction along with detailed descriptions in IAD domain., Comment: 8 pages, 7 figures

Inspired by certain optimization solvers, the deep unfolding network (DUN) has attracted much attention in recent years for image compressed sensing (CS). However, there still exist the following two issues: 1) In existing DUNs, most hyperparameters are usually content independent, which greatly limits their adaptability for different input contents. 2) In each iteration, a plain convolutional neural network is usually adopted, which weakens the perception of wider context prior and therefore depresses the expressive ability. In this paper, inspired by the traditional Proximal Gradient Descent (PGD) algorithm, a novel DUN for image compressed sensing (dubbed DUN-CSNet) is proposed to solve the above two issues. Specifically, for the first issue, a novel content adaptive gradient descent network is proposed, in which a well-designed step size generation sub-network is developed to dynamically allocate the corresponding step sizes for different textures of input image by generating a content-aware step size map, realizing a content-adaptive gradient updating. For the second issue, considering the fact that many similar patches exist in an image but have undergone a deformation, a novel deformation-invariant non-local proximal mapping network is developed, which can adaptively build the long-range dependencies between the nonlocal patches by deformation-invariant non-local modeling, leading to a wider perception on context priors. Extensive experiments manifest that the proposed DUN-CSNet outperforms existing state-of-the-art CS methods by large margins., Comment: 16 pages, 13 figures. Accepted by IEEE Transactions on Multimedia (TMM)

Reinforcement Learning (RL) and Imitation Learning (IL) have made great progress in robotic control in recent years. However, these methods show obvious deterioration for new tasks that need to be completed through new combinations of actions. RL methods heavily rely on reward functions that cannot generalize well for new tasks, while IL methods are limited by expert demonstrations which do not cover new tasks. In contrast, humans can easily complete these tasks with the fragmented knowledge learned from task-agnostic experience. Inspired by this observation, this paper proposes a task-agnostic learning method (TAL for short) that can learn fragmented knowledge from task-agnostic data to accomplish new tasks. TAL consists of four stages. First, the task-agnostic exploration is performed to collect data from interactions with the environment. The collected data is organized via a knowledge graph. Compared with the previous sequential structure, the knowledge graph representation is more compact and fits better for environment exploration. Second, an action feature extractor is proposed and trained using the collected knowledge graph data for task-agnostic fragmented knowledge learning. Third, a candidate action generator is designed, which applies the action feature extractor on a new task to generate multiple candidate action sets. Finally, an action proposal is designed to produce the probabilities for actions in a new task according to the environmental information. The probabilities are then used to select actions to be executed from multiple candidate action sets to form the plan. Experiments on a virtual indoor scene show that the proposed method outperforms the state-of-the-art offline RL method: CQL by 35.28% and the IL method: BC by 22.22%., Comment: 11 pages, 11 figures, Under Review

By integrating certain optimization solvers with deep neural network, deep unfolding network (DUN) has attracted much attention in recent years for image compressed sensing (CS). However, there still exist several issues in existing DUNs: 1) For each iteration, a simple stacked convolutional network is usually adopted, which apparently limits the expressiveness of these models. 2) Once the training is completed, most hyperparameters of existing DUNs are fixed for any input content, which significantly weakens their adaptability. In this paper, by unfolding the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA), a novel fast hierarchical DUN, dubbed FHDUN, is proposed for image compressed sensing, in which a well-designed hierarchical unfolding architecture is developed to cooperatively explore richer contextual prior information in multi-scale spaces. To further enhance the adaptability, series of hyperparametric generation networks are developed in our framework to dynamically produce the corresponding optimal hyperparameters according to the input content. Furthermore, due to the accelerated policy in FISTA, the newly embedded acceleration module makes the proposed FHDUN save more than 50% of the iterative loops against recent DUNs. Extensive CS experiments manifest that the proposed FHDUN outperforms existing state-of-the-art CS methods, while maintaining fewer iterations., Comment: Accepted by ACM MM 2022

Guided filter is a fundamental tool in computer vision and computer graphics which aims to transfer structure information from guidance image to target image. Most existing methods construct filter kernels from the guidance itself without considering the mutual dependency between the guidance and the target. However, since there typically exist significantly different edges in the two images, simply transferring all structural information of the guidance to the target would result in various artifacts. To cope with this problem, we propose an effective framework named deep attentional guided image filtering, the filtering process of which can fully integrate the complementary information contained in both images. Specifically, we propose an attentional kernel learning module to generate dual sets of filter kernels from the guidance and the target, respectively, and then adaptively combine them by modeling the pixel-wise dependency between the two images. Meanwhile, we propose a multi-scale guided image filtering module to progressively generate the filtering result with the constructed kernels in a coarse-to-fine manner. Correspondingly, a multi-scale fusion strategy is introduced to reuse the intermediate results in the coarse-to-fine process. Extensive experiments show that the proposed framework compares favorably with the state-of-the-art methods in a wide range of guided image filtering applications, such as guided super-resolution, cross-modality restoration, texture removal, and semantic segmentation.

Deep network-based image Compressed Sensing (CS) has attracted much attention in recent years. However, the existing deep network-based CS schemes either reconstruct the target image in a block-by-block manner that leads to serious block artifacts or train the deep network as a black box that brings about limited insights of image prior knowledge. In this paper, a novel image CS framework using non-local neural network (NL-CSNet) is proposed, which utilizes the non-local self-similarity priors with deep network to improve the reconstruction quality. In the proposed NL-CSNet, two non-local subnetworks are constructed for utilizing the non-local self-similarity priors in the measurement domain and the multi-scale feature domain respectively. Specifically, in the subnetwork of measurement domain, the long-distance dependencies between the measurements of different image blocks are established for better initial reconstruction. Analogically, in the subnetwork of multi-scale feature domain, the affinities between the dense feature representations are explored in the multi-scale space for deep reconstruction. Furthermore, a novel loss function is developed to enhance the coupling between the non-local representations, which also enables an end-to-end training of NL-CSNet. Extensive experiments manifest that NL-CSNet outperforms existing state-of-the-art CS methods, while maintaining fast computational speed., Comment: 14 pages, 11 figures, 7 tables

As a crucial part of video compression, intra prediction utilizes local information of images to eliminate the redundancy in spatial domain. In both the High Efficiency Video Coding (H.265/HEVC) and Versatile Video Coding (H.266/VVC), multiple directional prediction modes are employed to find the texture trend of each small block and then the prediction is made based on reference samples in the selected direction. Recently, the intra prediction schemes based on neural networks have achieved great success. In these methods, the networks are trained and applied to intra prediction to assist the directional prediction modes. In this paper, we propose a novel tree-structured data clustering-driven neural network (dubbed TreeNet) for intra prediction, which builds the networks and clusters the training data in a tree-structured manner. Specifically, in each network split and training process of TreeNet, every parent network on a leaf node is split into two child networks by adding or subtracting Gaussian random noise. Then a data clustering-driven training is applied to train the two derived child networks using the clustered training data of their parent. To test the performance, TreeNet is integrated into VVC and HEVC to combine with or replace the directional prediction modes. In addition, a fast termination strategy is proposed to accelerate the search of TreeNet. The experimental results demonstrate that TreeNet with the fast termination can reach an average of 2.8% Bjontegaard distortion rate (BD-rate) improvement (up to 8.1%) and 4.9% BD-rate improvement (up to 8.2%) over VVC (VTM-4.0) and HEVC (HM-16.9) with all intra configuration, respectively.

Depth map records distance between the viewpoint and objects in the scene, which plays a critical role in many real-world applications. However, depth map captured by consumer-grade RGB-D cameras suffers from low spatial resolution. Guided depth map super-resolution (DSR) is a popular approach to address this problem, which attempts to restore a high-resolution (HR) depth map from the input low-resolution (LR) depth and its coupled HR RGB image that serves as the guidance. The most challenging problems for guided DSR are how to correctly select consistent structures and propagate them, and properly handle inconsistent ones. In this paper, we propose a novel attention-based hierarchical multi-modal fusion (AHMF) network for guided DSR. Specifically, to effectively extract and combine relevant information from LR depth and HR guidance, we propose a multi-modal attention based fusion (MMAF) strategy for hierarchical convolutional layers, including a feature enhance block to select valuable features and a feature recalibration block to unify the similarity metrics of modalities with different appearance characteristics. Furthermore, we propose a bi-directional hierarchical feature collaboration (BHFC) module to fully leverage low-level spatial information and high-level structure information among multi-scale features. Experimental results show that our approach outperforms state-of-the-art methods in terms of reconstruction accuracy, running speed and memory efficiency.

The widespread application of audio communication technologies has speeded up audio data flowing across the Internet, which made it a popular carrier for covert communication. In this paper, we present a cross-modal steganography method for hiding image content into audio carriers while preserving the perceptual fidelity of the cover audio. In our framework, two multi-stage networks are designed: the first network encodes the decreasing multilevel residual errors inside different audio subsequences with the corresponding stage sub-networks, while the second network decodes the residual errors from the modified carrier with the corresponding stage sub-networks to produce the final revealed results. The multi-stage design of proposed framework not only make the controlling of payload capacity more flexible, but also make hiding easier because of the gradual sparse characteristic of residual errors. Qualitative experiments suggest that modifications to the carrier are unnoticeable by human listeners and that the decoded images are highly intelligible., Comment: ICASSP 2020

Mesh denoising is a critical technology in geometry processing that aims to recover high-fidelity 3D mesh models of objects from their noise-corrupted versions. In this work, we propose a learning-based normal filtering scheme for mesh denoising called NormalNet, which maps the guided normal filtering (GNF) into a deep network. The scheme follows the iterative framework of filtering-based mesh denoising. During each iteration, first, the voxelization strategy is applied on each face in a mesh to transform the irregular local structure into the regular volumetric representation, therefore, both the structure and face normal information are preserved and the convolution operations in CNN(Convolutional Neural Network) can be easily performed. Second, instead of the guidance normal generation and the guided filtering in GNF, a deep CNN is designed, which takes the volumetric representation as input, and outputs the learned filtered normals. At last, the vertex positions are updated according to the filtered normals. Specifically, the iterative training framework is proposed, in which the generation of training data and the network training are alternately performed, whereas the ground truth normals are taken as the guidance normals in GNF to get the target normals. Compared to state-of-the-art works, NormalNet can effectively remove noise while preserving the original features and avoiding pseudo-features.

Traditional intra prediction methods for HEVC rely on using the nearest reference lines for predicting a block, which ignore much richer context between the current block and its neighboring blocks and therefore cause inaccurate prediction especially when weak spatial correlation exists between the current block and the reference lines. To overcome this problem, in this paper, an intra prediction convolutional neural network (IPCNN) is proposed for intra prediction, which exploits the rich context of the current block and therefore is capable of improving the accuracy of predicting the current block. Meanwhile, the predictions of the three nearest blocks can also be refined. To the best of our knowledge, this is the first paper that directly applies CNNs to intra prediction for HEVC. Experimental results validate the effectiveness of applying CNNs to intra prediction and achieved significant performance improvement compared to traditional intra prediction methods., Comment: 10 pages, This is the extended edition of poster paper accepted by DCC 2017

The compressed sensing (CS) has been successfully applied to image compression in the past few years as most image signals are sparse in a certain domain. Several CS reconstruction models have been proposed and obtained superior performance. However, these methods suffer from blocking artifacts or ringing effects at low sampling ratios in most cases. To address this problem, we propose a deep convolutional Laplacian Pyramid Compressed Sensing Network (LapCSNet) for CS, which consists of a sampling sub-network and a reconstruction sub-network. In the sampling sub-network, we utilize a convolutional layer to mimic the sampling operator. In contrast to the fixed sampling matrices used in traditional CS methods, the filters used in our convolutional layer are jointly optimized with the reconstruction sub-network. In the reconstruction sub-network, two branches are designed to reconstruct multi-scale residual images and muti-scale target images progressively using a Laplacian pyramid architecture. The proposed LapCSNet not only integrates multi-scale information to achieve better performance but also reduces computational cost dramatically. Experimental results on benchmark datasets demonstrate that the proposed method is capable of reconstructing more details and sharper edges against the state-of-the-arts methods., Comment: 5 pages. Accepted by ICASSP2018

Gaussian random matrix (GRM) has been widely used to generate linear measurements in compressed sensing (CS) of natural images. However, there actually exist two disadvantages with GRM in practice. One is that GRM has large memory requirement and high computational complexity, which restrict the applications of CS. Another is that the CS measurements randomly obtained by GRM cannot provide sufficient reconstruction performances. In this paper, a Deep neural network based Sparse Measurement Matrix (DSMM) is learned by the proposed convolutional network to reduce the sampling computational complexity and improve the CS reconstruction performance. Two sub networks are included in the proposed network, which are the sampling sub-network and the reconstruction sub-network. In the sampling sub-network, the sparsity and the normalization are both considered by the limitation of the storage and the computational complexity. In order to improve the CS reconstruction performance, a reconstruction sub-network are introduced to help enhance the sampling sub-network. So by the offline iterative training of the proposed end-to-end network, the DSMM is generated for accurate measurement and excellent reconstruction. Experimental results demonstrate that the proposed DSMM outperforms GRM greatly on representative CS reconstruction methods, Comment: 5 pages, accepted by ICIP 2018

Soft-cast, a cross-layer design for wireless video transmission, is proposed to solve the drawbacks of digital video transmission: threshold transmission framework achieving the same effect. Specifically, in encoder, we carry out power allocation on the transformed coefficients and encode the coefficients based on the new formulation of power distortion. In decoder, the process of LLSE estimator is also improved. Accompanied with the inverse nonlinear transform, DCT coefficients can be recovered depending on the scaling factors , LLSE estimator coefficients and metadata. Experiment results show that our proposed framework outperforms the Soft-cast in PSNR 1.08 dB and the MSSIM gain reaches to 2.35% when transmitting under the same bandwidth and total power., Comment: 6 pages,7 figures

Deep learning, e.g., convolutional neural networks (CNNs), has achieved great success in image processing and computer vision especially in high level vision applications such as recognition and understanding. However, it is rarely used to solve low-level vision problems such as image compression studied in this paper. Here, we move forward a step and propose a novel compression framework based on CNNs. To achieve high-quality image compression at low bit rates, two CNNs are seamlessly integrated into an end-to-end compression framework. The first CNN, named compact convolutional neural network (ComCNN), learns an optimal compact representation from an input image, which preserves the structural information and is then encoded using an image codec (e.g., JPEG, JPEG2000 or BPG). The second CNN, named reconstruction convolutional neural network (RecCNN), is used to reconstruct the decoded image with high-quality in the decoding end. To make two CNNs effectively collaborate, we develop a unified end-to-end learning algorithm to simultaneously learn ComCNN and RecCNN, which facilitates the accurate reconstruction of the decoded image using RecCNN. Such a design also makes the proposed compression framework compatible with existing image coding standards. Experimental results validate that the proposed compression framework greatly outperforms several compression frameworks that use existing image coding standards with state-of-the-art deblocking or denoising post-processing methods., Comment: Submitted to IEEE Transactions on Circuits and Systems for Video Technology

The compressed sensing (CS) theory has been successfully applied to image compression in the past few years as most image signals are sparse in a certain domain. Several CS reconstruction models have been recently proposed and obtained superior performance. However, there still exist two important challenges within the CS theory. The first one is how to design a sampling mechanism to achieve an optimal sampling efficiency, and the second one is how to perform the reconstruction to get the highest quality to achieve an optimal signal recovery. In this paper, we try to deal with these two problems with a deep network. First of all, we train a sampling matrix via the network training instead of using a traditional manually designed one, which is much appropriate for our deep network based reconstruct process. Then, we propose a deep network to recover the image, which imitates traditional compressed sensing reconstruction processes. Experimental results demonstrate that our deep networks based CS reconstruction method offers a very significant quality improvement compared against state of the art ones., Comment: This paper has been accepted by the IEEE International Conference on Multimedia and Expo (ICME) 2017

Traditional works have shown that patches in a natural image tend to redundantly recur many times inside the image, both within the same scale, as well as across different scales. Make full use of these multi-scale information can improve the image restoration performance. However, the current proposed deep learning based restoration methods do not take the multi-scale information into account. In this paper, we propose a dilated convolution based inception module to learn multi-scale information and design a deep network for single image super-resolution. Different dilated convolution learns different scale feature, then the inception module concatenates all these features to fuse multi-scale information. In order to increase the reception field of our network to catch more contextual information, we cascade multiple inception modules to constitute a deep network to conduct single image super-resolution. With the novel dilated convolution based inception module, the proposed end-to-end single image super-resolution network can take advantage of multi-scale information to improve image super-resolution performance. Experimental results show that our proposed method outperforms many state-of-the-art single image super-resolution methods., Comment: This paper has been accepted by the IEEE International Conference on Image Processing (ICIP) 2017

Lossy image compression is generally formulated as a joint rate-distortion optimization to learn encoder, quantizer, and decoder. However, the quantizer is non-differentiable, and discrete entropy estimation usually is required for rate control. These make it very challenging to develop a convolutional network (CNN)-based image compression system. In this paper, motivated by that the local information content is spatially variant in an image, we suggest that the bit rate of the different parts of the image should be adapted to local content. And the content aware bit rate is allocated under the guidance of a content-weighted importance map. Thus, the sum of the importance map can serve as a continuous alternative of discrete entropy estimation to control compression rate. And binarizer is adopted to quantize the output of encoder due to the binarization scheme is also directly defined by the importance map. Furthermore, a proxy function is introduced for binary operation in backward propagation to make it differentiable. Therefore, the encoder, decoder, binarizer and importance map can be jointly optimized in an end-to-end manner by using a subset of the ImageNet database. In low bit rate image compression, experiments show that our system significantly outperforms JPEG and JPEG 2000 by structural similarity (SSIM) index, and can produce the much better visual result with sharp edges, rich textures, and fewer artifacts.

Given the prevalence of JPEG compressed images, optimizing image reconstruction from the compressed format remains an important problem. Instead of simply reconstructing a pixel block from the centers of indexed DCT coefficient quantization bins (hard decoding), soft decoding reconstructs a block by selecting appropriate coefficient values within the indexed bins with the help of signal priors. The challenge thus lies in how to define suitable priors and apply them effectively. In this paper, we combine three image priors---Laplacian prior for DCT coefficients, sparsity prior and graph-signal smoothness prior for image patches---to construct an efficient JPEG soft decoding algorithm. Specifically, we first use the Laplacian prior to compute a minimum mean square error (MMSE) initial solution for each code block. Next, we show that while the sparsity prior can reduce block artifacts, limiting the size of the over-complete dictionary (to lower computation) would lead to poor recovery of high DCT frequencies. To alleviate this problem, we design a new graph-signal smoothness prior (desired signal has mainly low graph frequencies) based on the left eigenvectors of the random walk graph Laplacian matrix (LERaG). Compared to previous graph-signal smoothness priors, LERaG has desirable image filtering properties with low computation overhead. We demonstrate how LERaG can facilitate recovery of high DCT frequencies of a piecewise smooth (PWS) signal via an interpretation of low graph frequency components as relaxed solutions to normalized cut in spectral clustering. Finally, we construct a soft decoding algorithm using the three signal priors with appropriate prior weights. Experimental results show that our proposal outperforms state-of-the-art soft decoding algorithms in both objective and subjective evaluations noticeably.

Image interpolation is one of the most elementary imaging research topics. A number of image interpolation methods have been developed and tested on uncompressed images in the literature. However, a lot of videos have already been stored or have to be transmitted in compressed format due to the storage limitation or the bandwidth limitation. The existed image interpolation methods may not be efficient when directly applied to compressed images or videos. Inspired by the success of the intra prediction in HEVC and the edge-directed image interpolation methods, a directional intra frame interpolation for HEVC compressed video is proposed. The main idea is to use the directional prediction information in compressed low-resolution video bitstreams to estimate the associated high-resolution video. For intra frames, the prediction direction information is taken into account as context in the directional interpolation. When a pixel is decompressed with a small prediction residual, the interpolation is performed along its block direction. The interpolation weight for each block direction is off-line trained by the Wiener filter based on the representative video sequences. For each pixel with a large prediction residual, a piecewise autoregressive model is used as a regularization term into the interpolation function. Extensive experiments demonstrate that the proposed method achieves better performance than the traditional methods such as Bicubic, KR, LAZA, NEDI and SAI.

Wireless video broadcast poses a challenge to the conventional visual communication in providing simultaneously each receiver the best video quality under its channel condition. Soft video broadcast, as a newly emerged wireless video broadcast scheme, is able to accommodate multiple receivers of different channel SNR. However, the current soft video broadcast frameworks such as SoftCast require the bandwidth of wireless channel to match the number of video coefficients per second. When the channel bandwidth is larger, the existing frameworks become not very efficient in bandwidth expansion. More importantly, it is possible that the users in broadcast applications have different bandwidth. However, none of the existing soft video broadcast frameworks considers bandwidth heterogeneity. In this paper, we propose a soft video broadcast framework, called LayerCast, which can accommodate simultaneously heterogeneous users with diverse SNR and diverse bandwidth. The bandwidth expansion problem is solved by applying layered coset coding. More importantly, we derive globally optimal power allocation between layers and, within each layer, between each DCT chunk. In simulations, the proposed framework outperforms SoftCast of up to 4dB in video PSNR, and outperforms H.264-based framework up to 8dB in broadcast.

Image interpolation is one of the most elementary imaging research topics. A number of image interpolation methods have been developed and tested on uncompressed images in the literature. However, a lot of videos have already been stored or have to be transmitted in compressed format due to the storage limitation or the bandwidth limitation. The existed image interpolation methods may not be efficient when directly applied to compressed images or videos. Inspired by the success of the intra prediction in HEVC and the edge-directed image interpolation methods, a directional intra frame interpolation for HEVC compressed video is proposed. The main idea is to use the directional prediction information in compressed low-resolution video bitstreams to estimate the associated high-resolution video. For intra frames, the prediction direction information is taken into account as context in the directional interpolation. When a pixel is decompressed with a small prediction residual, the interpolation is performed along its block direction. The interpolation weight for each block direction is off-line trained by the Wiener filter based on the representative video sequences. For each pixel with a large prediction residual, a piecewise autoregressive model is used as a regularization term into the interpolation function. Extensive experiments demonstrate that the proposed method achieves better performance than the traditional methods such as Bicubic, KR, LAZA, NEDI and SAI.

Wireless video broadcast poses a challenge to the conventional visual communication in providing simultaneously each receiver the best video quality under its channel condition. Soft video broadcast, as a newly emerged wireless video broadcast scheme, is able to accommodate multiple receivers of different channel SNR. However, the current soft video broadcast frameworks such as SoftCast require the bandwidth of wireless channel to match the number of video coefficients per second. When the channel bandwidth is larger, the existing frameworks become not very efficient in bandwidth expansion. More importantly, it is possible that the users in broadcast applications have different bandwidth. However, none of the existing soft video broadcast frameworks considers bandwidth heterogeneity. In this paper, we propose a soft video broadcast framework, called LayerCast, which can accommodate simultaneously heterogeneous users with diverse SNR and diverse bandwidth. The bandwidth expansion problem is solved by applying layered coset coding. More importantly, we derive globally optimal power allocation between layers and, within each layer, between each DCT chunk. In simulations, the proposed framework outperforms SoftCast of up to 4dB in video PSNR, and outperforms H.264-based framework up to 8dB in broadcast.

Image interpolation is one of the most elementary imaging research topics. A number of image interpolation methods have been developed and tested on uncompressed images in the literature. However, a lot of videos have already been stored or have to be transmitted in compressed format due to the storage limitation or the bandwidth limitation. The existed image interpolation methods may not be efficient when directly applied to compressed images or videos. Inspired by the success of the intra prediction in HEVC and the edge-directed image interpolation methods, a directional intra frame interpolation for HEVC compressed video is proposed. The main idea is to use the directional prediction information in compressed low-resolution video bitstreams to estimate the associated high-resolution video. For intra frames, the prediction direction information is taken into account as context in the directional interpolation. When a pixel is decompressed with a small prediction residual, the interpolation is performed along its block direction. The interpolation weight for each block direction is off-line trained by the Wiener filter based on the representative video sequences. For each pixel with a large prediction residual, a piecewise autoregressive model is used as a regularization term into the interpolation function. Extensive experiments demonstrate that the proposed method achieves better performance than the traditional methods such as Bicubic, KR, LAZA, NEDI and SAI.

Wireless video broadcast poses a challenge to the conventional visual communication in providing simultaneously each receiver the best video quality under its channel condition. Soft video broadcast, as a newly emerged wireless video broadcast scheme, is able to accommodate multiple receivers of different channel SNR. However, the current soft video broadcast frameworks such as SoftCast require the bandwidth of wireless channel to match the number of video coefficients per second. When the channel bandwidth is larger, the existing frameworks become not very efficient in bandwidth expansion. More importantly, it is possible that the users in broadcast applications have different bandwidth. However, none of the existing soft video broadcast frameworks considers bandwidth heterogeneity. In this paper, we propose a soft video broadcast framework, called LayerCast, which can accommodate simultaneously heterogeneous users with diverse SNR and diverse bandwidth. The bandwidth expansion problem is solved by applying layered coset coding. More importantly, we derive globally optimal power allocation between layers and, within each layer, between each DCT chunk. In simulations, the proposed framework outperforms SoftCast of up to 4dB in video PSNR, and outperforms H.264-based framework up to 8dB in broadcast.

High Efficiency Video Coding (HEVC) is the emerging video coding standard, which provides equivalent subjective quality with about 50% bit rate reduction compared to H.264/AVC High Profile. However, the improvement of coding efficiency is obtained at the expense of increased computational complexity. In this paper, a fast algorithm for HEVC intra coding is proposed. Firstly, a depth range prediction method by exploiting the correlation between the neighboring Coding Tree Units (CTUs) is proposed. Secondly, the rate distortion costs and Sum of Absolute Difference with Hadamard transform (HSAD) of recently encoded CUs are used to decide whether current CU will be further divided or not. Finally, only the intra prediction modes (IPMs) with lower precision are employed in the rough mode decision (RMD) and IPMs for rate distortion optimization (RDO) are reduced based on the correlation between neighboring CUs. Experimental results show that our proposed algorithm can achieve average 54% (up to 57%) encoding time saving while causing negligible RD performance loss (1.0% BD-rate increase on average) in All Intra High Efficiency test condition compared with HM 10.0.

The emerging High Efficiency Video Coding (HEVC) standard provides equivalent subjective quality with about 50% bit rate reduction compared to the H.264/AVC High profile. However, the improvement of coding efficiency is obtained at the expense of increased computational complexity. This paper presents a fast intra-encoding algorithm for HEVC, which is composed of the following four techniques. Firstly, an early termination technique for coding unit (CU) depth decision is proposed based on the depth of neighboring CUs and the comparison results of rate distortion (RD) costs between the parent CU and part of its child CUs. Secondly, the correlation of intra-prediction modes between neighboring PUs is exploited to accelerate the intra-prediction mode decision for HEVC intra-coding and the impact of the number of mode candidates after the rough mode decision (RMD) process in HM is studied in our work. Thirdly, the TU depth range is restricted based on the probability of each TU depth and one redundant process is removed in the TU depth selection process based on the analysis of the HEVC reference software. Finally, the probability of each case for the intra-transform skip mode is studied to accelerate the intra-transform skip mode decision. Experimental results show that the proposed algorithm can provide about 50% time savings with only 0.5% BD-rate loss on average when compared to HM 11.0 for the Main profile all-intra-configuration. Parts of these techniques have been adopted into the HEVC reference software. © 2014 Elsevier B.V.

The emerging High Efficiency Video Coding (HEVC) standard provides equivalent subjective quality with about 50% bit rate reduction compared to the H.264/AVC High profile. However, the improvement of coding efficiency is obtained at the expense of increased computational complexity. This paper presents a fast intra-encoding algorithm for HEVC, which is composed of the following four techniques. Firstly, an early termination technique for coding unit (CU) depth decision is proposed based on the depth of neighboring CUs and the comparison results of rate distortion (RD) costs between the parent CU and part of its child CUs. Secondly, the correlation of intra-prediction modes between neighboring PUs is exploited to accelerate the intra-prediction mode decision for HEVC intra-coding and the impact of the number of mode candidates after the rough mode decision (RMD) process in HM is studied in our work. Thirdly, the TU depth range is restricted based on the probability of each TU depth and one redundant process is removed in the TU depth selection process based on the analysis of the HEVC reference software. Finally, the probability of each case for the intra-transform skip mode is studied to accelerate the intra-transform skip mode decision. Experimental results show that the proposed algorithm can provide about 50% time savings with only 0.5% BD-rate loss on average when compared to HM 11.0 for the Main profile all-intra-configuration. Parts of these techniques have been adopted into the HEVC reference software. © 2014 Elsevier B.V.

High Efficiency Video Coding (HEVC) is the emerging video coding standard, which provides equivalent subjective quality with about 50% bit rate reduction compared to H.264/AVC High Profile. However, the improvement of coding efficiency is obtained at the expense of increased computational complexity. In this paper, a fast algorithm for HEVC intra coding is proposed. Firstly, a depth range prediction method by exploiting the correlation between the neighboring Coding Tree Units (CTUs) is proposed. Secondly, the rate distortion costs and Sum of Absolute Difference with Hadamard transform (HSAD) of recently encoded CUs are used to decide whether current CU will be further divided or not. Finally, only the intra prediction modes (IPMs) with lower precision are employed in the rough mode decision (RMD) and IPMs for rate distortion optimization (RDO) are reduced based on the correlation between neighboring CUs. Experimental results show that our proposed algorithm can achieve average 54% (up to 57%) encoding time saving while causing negligible RD performance loss (1.0% BD-rate increase on average) in All Intra High Efficiency test condition compared with HM 10.0.

The emerging High Efficiency Video Coding (HEVC) standard provides equivalent subjective quality with about 50% bit rate reduction compared to the H.264/AVC High profile. However, the improvement of coding efficiency is obtained at the expense of increased computational complexity. This paper presents a fast intra-encoding algorithm for HEVC, which is composed of the following four techniques. Firstly, an early termination technique for coding unit (CU) depth decision is proposed based on the depth of neighboring CUs and the comparison results of rate distortion (RD) costs between the parent CU and part of its child CUs. Secondly, the correlation of intra-prediction modes between neighboring PUs is exploited to accelerate the intra-prediction mode decision for HEVC intra-coding and the impact of the number of mode candidates after the rough mode decision (RMD) process in HM is studied in our work. Thirdly, the TU depth range is restricted based on the probability of each TU depth and one redundant process is removed in the TU depth selection process based on the analysis of the HEVC reference software. Finally, the probability of each case for the intra-transform skip mode is studied to accelerate the intra-transform skip mode decision. Experimental results show that the proposed algorithm can provide about 50% time savings with only 0.5% BD-rate loss on average when compared to HM 11.0 for the Main profile all-intra-configuration. Parts of these techniques have been adopted into the HEVC reference software. © 2014 Elsevier B.V.

High Efficiency Video Coding (HEVC) is the emerging video coding standard, which provides equivalent subjective quality with about 50% bit rate reduction compared to H.264/AVC High Profile. However, the improvement of coding efficiency is obtained at the expense of increased computational complexity. In this paper, a fast algorithm for HEVC intra coding is proposed. Firstly, a depth range prediction method by exploiting the correlation between the neighboring Coding Tree Units (CTUs) is proposed. Secondly, the rate distortion costs and Sum of Absolute Difference with Hadamard transform (HSAD) of recently encoded CUs are used to decide whether current CU will be further divided or not. Finally, only the intra prediction modes (IPMs) with lower precision are employed in the rough mode decision (RMD) and IPMs for rate distortion optimization (RDO) are reduced based on the correlation between neighboring CUs. Experimental results show that our proposed algorithm can achieve average 54% (up to 57%) encoding time saving while causing negligible RD performance loss (1.0% BD-rate increase on average) in All Intra High Efficiency test condition compared with HM 10.0.

Traditional patch-based sparse representation modeling of natural images usually suffer from two problems. First, it has to solve a large-scale optimization problem with high computational complexity in dictionary learning. Second, each patch is considered independently in dictionary learning and sparse coding, which ignores the relationship among patches, resulting in inaccurate sparse coding coefficients. In this paper, instead of using patch as the basic unit of sparse representation, we exploit the concept of group as the basic unit of sparse representation, which is composed of nonlocal patches with similar structures, and establish a novel sparse representation modeling of natural images, called group-based sparse representation (GSR). The proposed GSR is able to sparsely represent natural images in the domain of group, which enforces the intrinsic local sparsity and nonlocal self-similarity of images simultaneously in a unified framework. Moreover, an effective self-adaptive dictionary learning method for each group with low complexity is designed, rather than dictionary learning from natural images. To make GSR tractable and robust, a split Bregman based technique is developed to solve the proposed GSR-driven minimization problem for image restoration efficiently. Extensive experiments on image inpainting, image deblurring and image compressive sensing recovery manifest that the proposed GSR modeling outperforms many current state-of-the-art schemes in both PSNR and visual perception., Comment: 34 pages, 6 tables, 19 figures, to be published in IEEE Transactions on Image Processing; Project, Code and High resolution PDF version can be found: http://idm.pku.edu.cn/staff/zhangjian/. arXiv admin note: text overlap with arXiv:1404.7566

This paper presents a novel strategy for high-fidelity image restoration by characterizing both local smoothness and nonlocal self-similarity of natural images in a unified statistical manner. The main contributions are three-folds. First, from the perspective of image statistics, a joint statistical modeling (JSM) in an adaptive hybrid space-transform domain is established, which offers a powerful mechanism of combining local smoothness and nonlocal self-similarity simultaneously to ensure a more reliable and robust estimation. Second, a new form of minimization functional for solving image inverse problem is formulated using JSM under regularization-based framework. Finally, in order to make JSM tractable and robust, a new Split-Bregman based algorithm is developed to efficiently solve the above severely underdetermined inverse problem associated with theoretical proof of convergence. Extensive experiments on image inpainting, image deblurring and mixed Gaussian plus salt-and-pepper noise removal applications verify the effectiveness of the proposed algorithm., Comment: 14 pages, 18 figures, 7 Tables, to be published in IEEE Transactions on Circuits System and Video Technology (TCSVT). High resolution pdf version and Code can be found at: http://idm.pku.edu.cn/staff/zhangjian/IRJSM

A novel coding strategy for block-based compressive sens-ing named spatially directional predictive coding (SDPC) is proposed, which efficiently utilizes the intrinsic spatial cor-relation of natural images. At the encoder, for each block of compressive sensing (CS) measurements, the optimal pre-diction is selected from a set of prediction candidates that are generated by four designed directional predictive modes. Then, the resulting residual is processed by scalar quantiza-tion (SQ). At the decoder, the same prediction is added onto the de-quantized residuals to produce the quantized CS measurements, which is exploited for CS reconstruction. Experimental results substantiate significant improvements achieved by SDPC-plus-SQ in rate distortion performance as compared with SQ alone and DPCM-plus-SQ., Comment: 5 pages, 3 tables, 3 figures, published at IEEE International Conference on Image Processing (ICIP) 2013 Code Avaiable: http://idm.pku.edu.cn/staff/zhangjian/SDPC

Compressive Sensing (CS) theory shows that a signal can be decoded from many fewer measurements than suggested by the Nyquist sampling theory, when the signal is sparse in some domain. Most of conventional CS recovery approaches, however, exploited a set of fixed bases (e.g. DCT, wavelet, contourlet and gradient domain) for the entirety of a signal, which are irrespective of the nonstationarity of natural signals and cannot achieve high enough degree of sparsity, thus resulting in poor rate-distortion performance. In this paper, we propose a new framework for image compressive sensing recovery via structural group sparse representation (SGSR) modeling, which enforces image sparsity and self-similarity simultaneously under a unified framework in an adaptive group domain, thus greatly confining the CS solution space. In addition, an efficient iterative shrinkage/thresholding algorithm based technique is developed to solve the above optimization problem. Experimental results demonstrate that the novel CS recovery strategy achieves significant performance improvements over the current state-of-the-art schemes and exhibits nice convergence., Comment: 10 pages, 4 figures, 1 table, published at IEEE Data Compression Conference (DCC) 2013

From many fewer acquired measurements than suggested by the Nyquist sampling theory, compressive sensing (CS) theory demonstrates that, a signal can be reconstructed with high probability when it exhibits sparsity in some domain. Most of the conventional CS recovery approaches, however, exploited a set of fixed bases (e.g. DCT, wavelet and gradient domain) for the entirety of a signal, which are irrespective of the non-stationarity of natural signals and cannot achieve high enough degree of sparsity, thus resulting in poor CS recovery performance. In this paper, we propose a new framework for image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization. The intrinsic sparsity of natural images is enforced substantially by sparsely representing overlapped image patches using the adaptively learned sparsifying basis in the form of L0 norm, greatly reducing blocking artifacts and confining the CS solution space. To make our proposed scheme tractable and robust, a split Bregman iteration based technique is developed to solve the non-convex L0 minimization problem efficiently. Experimental results on a wide range of natural images for CS recovery have shown that our proposed algorithm achieves significant performance improvements over many current state-of-the-art schemes and exhibits good convergence property., Comment: 31 pages, 4 tables, 12 figures, to be published at Signal Processing, Code available: http://idm.pku.edu.cn/staff/zhangjian/ALSB

Recent years we have witnessed a striking rise of the compound video, which is generated by computers. The High Efficiency Video Coding standard (HEVC) adopts the intra transform skip mode to improve the coding efficiency of compound video. In this paper, we improve the intra transform skip mode from two aspects. Firstly, we study the statistical character of the intra prediction residual and propose a separate coefficient scan order for the intra transform skip mode. Secondly, we analyze the probability of the intra transform skip mode for different coding unit (CU) sizes and prediction unit (PU) sizes, and then propose one limitation to reduce the applied range of intra transform skip mode. The experimental results show that the first method can achieve 1.6%, 1.1%, and 0.7% luma coding gains on average in terms of BD-rate for class F under HE10-AI, HE10-RA, and HE10-LB conditions respectively; the second method can bring 9% encoder run time reduction with almost the same BD-rate changes. The proposed methods have been partially adopted into HEVC standard. © 2013 IEEE.

In recent years, video broadcast has become a popular application, but the traditional hierarchical design requires the source to pick a bitrate and video resolution for encoding before transmission, it cannot be efficient to accommodate users with different channel quality. The proposed DCAST scheme can solve this problem. DCAST uses the ME and MC technology to generate a predicted frame which helps current frame to do coset coding. However, in the ME process, DCAST uses fixed block size. In this paper, we use the variable block size motion estimation to replace the fixed block size motion estimation, it can effectively reduce the block effect and improve the quality of the reconstructed frame generated by MC and predicted frame. The DCAST with variable block size motion estimation is 0.5dB better than DCAST with fixed block size motion estimation. © 2013 IEEE.

In this paper, we propose a full reference Stereoscopic Video Quality Assessment (SVQA) algorithm based on the Stereo Just-Noticeable Difference (SJND) model. Firstly, SJND mimic the human binocular visual system characteristics from four factors, including: sensitivity of luminance contrast, spatial masking, temporal masking and binocular masking. Secondly, based on the SJND model, the full reference SVQA is developed, by capturing spatio-temporal distortions and binocular perceptions. Finally, experimental results have demonstrated that the proposed SVQA outperforms other four current evaluation methods and has a good consistency with the observers' subjective perception. © 2013 IEEE.

With the rapid development of digital video technology, frame rate up conversion is widely used. In this paper, a novel motion vector refinement method for frame rate up conversion on depth based 3D video is proposed. Our method involves two major stages in frame rate up conversion which are motion estimation and motion vector filtering. In the motion estimation process, the depth constraint to block matching algorithm is introduced in the bi-directional motion estimation method to obtain the motion vectors. In the motion vector filtering process, a depth-guided filter is designed to enhance the consistence of motions in the same depth plane. The refined motion vectors are used for frame interpolation. Experimental results show that the proposed method achieves 0.45 dB gain in terms of PSNR on average and improves the visual quality of the frame rate up-converted video. © 2013 IEEE.

In this paper, we propose to estimate the end-to-end distortion of virtual view for error-resilient depth map coding by analyzing the view-rendering process. The end-to-end distortion is estimated according to the expectation of depth distortion. The expectation of depth distortion is estimated based on the error-propagated distortion from the reference frame, which is calculated recursively for each pixel by considering the source characteristics, channel condition and error concealment method. The error-propagated distortion in terms of each frame is updated after the frame is encoded, which is used for the subsequent frames. Experimental results demonstrate that the proposed algorithm achieves substantial and consistent gains in comparison with the random intra updating method and the conventional rate-distortion method for error-resilient depth map coding. © 2013 IEEE.

Video broadcasting is a popular application of wireless network, whose main challenge is to accommodate different users with different channel conditions. Recently, a novel 'D-Cast' approach based on distributed source coding (DSC) is proposed. It can avoid error propagation and still achieve high compression efficiency in inter frame coding by utilizing coset coding and soft broadcast. However, D-CAST is not very efficient because of rough side information. In this work, we present a novel soft mobile video broadcast approach based on side information refinement algorithm (SIR-CAST) to improve the quality of the side information. Moreover, SIR-Cast optimizes the estimate of the quantifying step (Qstep) which is corresponding to the refined side information. Thus, SIR-CAST outperforms D-CAST about 1dB-2dB in video PSNR while maintaining the similar graceful degradation feature as D-CAST. © 2013 IEEE.

Error robustness becomes an important issue when the compressed depth map is transmitted over error prone network. There have been some algorithms to improve the error robustness of color video in the past years. However, their extensions to depth map are not reasonable due to the difference between depth map and color video. In this paper, a novel error robust distortion model is proposed to enhance the error robustness of depth map, in which the end-to-end distortion of virtual view is estimated. More specifically, the proposed distortion model recursively computes the expected decoded depth for each pixel by considering the channel condition and error concealment method, and then the expected decoded depth is used to estimate the end-to-end distortion of virtual view. Experimental results show that the proposed distortion model consistently outperforms the conventional distortion model and outperforms the random intra updating algorithm in most cases. © 2013 IEEE.

In the High Efficiency Video Coding (HEVC) based 3D video coding, 3D-HEVC, the disparity vector (DV) derivation is critical for inter-view motion prediction, inter-view residual prediction, disparity-compensated prediction (DCP) or any other tools exploiting inter-view correlation. In HTM-5.0.1, the DV is derived from some spatial and temporal neighbors to locate a corresponding block in another view. This paper advocates modifications of the DV derivation to reduce the complexity and to achieve slightly better coding efficiency. Firstly, we propose to remove the additional temporal block, unify the DV searching order for all views, and to impose restrictions on the temporal blocks for memory access bandwidth and complexity reduction in DV derivation. We also propose an improved DV searching order for slightly better coding performance. These proposed methods were adopted into the 3D-HEVC standard in the 3rd JCT-3V meeting in Jan. 2013. © 2013 IEEE.

Traditional video compression methods remove spatial and temporal redundancy based on the statistical correlation. However the final receptor is the human, we can remove the perception redundancy to get higher compression efficiency, by taking use of the properties of human visual system (HVS). Research has simulated the sensitivity of HVS to luminance contrast and spatial and temporal masking effects with the just-noticeable-distortion (JND) model, which describes the perception redundancy quantitatively. This paper proposes a new model named MJND (JND in Multi-view), which explores the property of HVS to stereoscopic masking effect. The proposed model not only contains the spatial and temporal JND, but also includes the JND in depth. The MJND model is then used in macroblock (MB) quantization adjustment and rate-distortion optimization in multi-view video coding (MVC). Compared with the standard MVC scheme without JND, our model can get better visual quality in the case of the same bit rate.

One important problem in distributed video coding is to estimate the variance of the correlation noise between the video signal and its decoder side information. This variance is hard to estimate due to the lack of the motion vectors at the encoder side. In this paper, we first propose a linear model to estimate this variance by referring the zero motion prediction at the encoder based on a Markov field assumption. Furthermore, not only the prediction noise from the video signal itself but also the additional noise due to wireless transmission is considered in this paper. We applied our correlation estimation method in our recent distributed wireless visual communication framework called DCAST. The experimental results show that the proposed method improves the video PSNR by 0.5-1.5dB while avoiding motion estimation at encoder. © 2013 IEEE.