Search

Consider a Gaussian memoryless multiple source (GMMS) with $m$ components with joint probability distribution known only to lie in a given class of distributions. A subset of $k \leq m$ components is sampled and compressed with the objective of reconstructing all the $m$ components within a specified level of distortion under a mean-squared error criterion. In Bayesian and nonBayesian settings, the notion of universal sampling rate-distortion function for Gaussian sources is introduced to capture the optimal tradeoffs among sampling, compression rate, and distortion level. Single-letter characterizations are provided for the universal sampling rate-distortion function. Our achievability proofs highlight the following structural property: it is optimal to compress and reconstruct first the sampled components of the GMMS alone, and then form estimates for the unsampled components based on the former.

Blau and Michaeli recently introduced a novel concept for inverse problems of signal processing, that is, the perception-distortion tradeoff. We introduce their tradeoff into the rate distortion theory of variable-length lossy source coding in information theory, and clarify the tradeoff among information rate, distortion and perception for general information sources. We also discuss the fixed-length coding with average distortion criterion that was missing in the previous letter., Comment: comex.cls, 5 pages, no figure. This is a continuation of the previous letter arXiv:1808.07986

Biological organisms are open, adaptve systems that can respond to changes in environment in specific ways. Adaptation and response can be posed as an optimization problem, with a tradeoff between the benefit obtained from a response and the cost of producing environment-specific responses. Using recent results in stochastic thermodynamics, we formulate the cost as the mutual information between the environment and the stochastic response. The problem of designing an optimally performing network now reduces to a problem in rate distortion theory -- a branch of information theory that deals with lossy data compression. We find that as the cost of unit information goes down, the system undergoes a sequence of transitions, corresponding to the recruitment of an increasing number of responses, thus improving response specificity as well as the net payoff. We derive formal equations for the transition points and exactly solve them for special cases. The first transition point, also called the {\it coding transition}, demarcates the boundary between a passive response and an active decision-making by the system. We study this transition point in detail, and derive three classes of asymptotic behavior, corresponding to the three limiting distributions of the statistics of extreme values. Our work points to the necessity of a union between information theory and the theory of adaptive biomolecular networks, in particular metabolic networks., Comment: 15 pages, 8 figures

This paper proposes the novel model for estima- tion of bounds in digital coding of images. Entropy coding of images is exploited to measure the useful information content of the data. The bit rate achieved by reversible compression using the rate-distortion theory approach takes into account the contribution of the observation noise and the intrinsic information of hypothetical noise-free image. Assuming the Laplacian probability density func- tion of the quantizer input signal, SQNR gains are calcu- lated for image predictive coding system with non-adaptive quantizer for white and correlated noise, respectively. The proposed model is evaluated on seabed images. However, model presented in this paper can be applied to any signal with Laplacian distribution.

The High Efficiency Video Coding standard provides improved compression ratio in comparison with its predecessors at the cost of large increases in the encoding computational complexity. An important share of this increase is due to the new flexible partitioning structures, namely the coding trees, the prediction units, and the residual quadtrees, with the best configurations decided through an exhaustive rate-distortion optimization (RDO) process. In this paper, we propose a set of procedures for deciding whether the partition structure optimization algorithm should be terminated early or run to the end of an exhaustive search for the best configuration. The proposed schemes are based on decision trees obtained through data mining techniques. By extracting intermediate data, such as encoding variables from a training set of video sequences, three sets of decision trees are built and implemented to avoid running the RDO algorithm to its full extent. When separately implemented, these schemes achieve average computational complexity reductions (CCRs) of up to 50% at a negligible cost of 0.56% in terms of Bjontegaard Delta (BD) rate increase. When the schemes are jointly implemented, an average CCR of up to 65% is achieved, with a small BD-rate increase of 1.36%. Extensive experiments and comparisons with similar works demonstrate that the proposed early termination schemes achieve the best rate-distortion-complexity tradeoffs among all the compared works.

In three dimensional video coding (3-DVC), it is reasonable to allocate bits for texture videos and depth maps differently as the characteristic of texture videos is different with that of depth maps. In order to improve the accuracy of bit allocation performance for 3-DVC, a more accurate distortion model for virtual view and rate models for texture videos and depth maps are proposed based on interview rate distortion analysis. Based on the proposed models, the bit allocation optimization problem is resolved by a coarse to fine strategy. The proposed bit allocation algorithm is implemented in the High Efficiency Video Coding-based 3-DVC platform version 6.0 (3-D-HTM 6.0). Experimental results demonstrate that the proposed distortion and rate models are accurate; meanwhile, the absolute difference (denoted as “rate inaccuracy”) between the actual coding bit rate ( ${R} _{\textbf {ABR}}$ ) and the target coding bit rate ( ${R} _{\textbf {TBR}}$ ) of the proposed method is only 2.67% on average; while the rate inaccuracy of the existing planar model-based method is 17.71%. Besides, there is an average 29.68% complexity increment when comparing with the planar model-based method.

High-efficiency video coding (HEVC) outperforms H.264/AVC by providing a bitrate reduction of about 50% while having almost the same perceptual quality. It adopts more flexible partitioning in motion estimation (ME) which gains higher coding efficiency at a cost of increased coding complexity. Depth maps in the emerging multi-view plus depth videos are exploited to adjust the search range in ME for HEVC complexity reduction. With the aid of depth intensity variations among neighbouring blocks, the proposed algorithm derives weights to the neighbouring blocks and establishes an adaptive search range according to the weighted sum of the motion vectors from the neighbouring blocks. Compared with the fast test zone (TZS) search in HEVC, the proposed algorithm saves 65% coding time in ME on average with insignificant rate-distortion degradation.

In this work, we use a multi-agent learning framework based on information theory concepts such as maximum entropy and rate distortion theory in order to control the reactive power sharing in an islanded microgrid. The distortion between each agent (distributed generator) and the environment behavior determines a fitness function that is used as a signal to control reactive power sharing in the system. The results, which were obtained through a simulated four nodes microgrid, show how the reactive power is shared in the system, specially when considerable load variations are present.

Rate-distortion optimization has been a successful and significant method in video coding. By introducing Lagrange multiplier optimization into compress procedure, we can choose coding parameters simply and effectively. In nearest neighbor search problem, hashing has been a popular method to reduce computation and storage cost, which is consistent with video coding method. Conventionally, we evaluate a hashing method with mAP (mean average precision) w.r.t. different bit number, but leave bit cost as an independent measure index. In this paper, we make an attempt to combine retrieval accuracy and bit cost to make evaluation more comprehensive, using the concept of rate distortion optimization. Consequently, we obtain an evaluation criterion to judge which work point of a specific hashing method is better, taking both the accuracy and the bit cost into account. The exertion of an algorithm can be then determined.

This paper addresses the problem of designing a global tone mapping operator for rate distortion optimized backward compatible compression of high dynamic range (HDR) images. We address the problem of tone mapping design for two different use cases leading to two different minimization problems. The first problem considered is the minimization of the distortion on the reconstructed HDR signal under a rate constraint on the standard dynamic range (SDR) layer. The second problem remains the same minimization with an additional constraint to preserve a good quality for the SDR signal. Both the distortion and rate are expressed as a function of the spatial gradient in HDR images. Experiments show that the proposed rate and distortion models based on the HDR image gradient accurately predict the real image rate and distortion measures. Experimental results show that for the first minimization, the optimal rate-distortion performances are achieved, and that the second optimization yields the best tradeoff between rate-distortion performance and quality preservation of the SDR signal.

Wavelet-based data compression methods are widely used for the data compression in power systems. There are two major problems of wavelet-based data compression techniques: there is no standard for the selection of best wavelet functions and decomposition scales; most of the existing wavelet-based data compression methods are only suitable for disturbance signals with abrupt change. This paper proposed a wavelet-based universal data compression method for different types of signals in power systems, including oscillations, such as low-frequency oscillations (LFOs) and sub-synchronous oscillations (SSOs), and disturbances. Based on compression ratios (CRs) and distortion rates (DRs), this paper constructed a compression distortion composite index (CDCI) for the evaluation of compression methods, with a balanced consideration of compression and accuracy. Based on CDCI, the best wavelet functions and decomposition scales for different signals can be selected from the candidates. The universality of the proposed method was verified by analyzing the compression of different typical actual recorded signals. The results indicated that the proposed method can provide high compression ratios and low distortion rates.

We examine the coordinated and universal rate-efficient sampling of a subset of correlated discrete memoryless sources followed by lossy compression of the sampled sources. The goal is to reconstruct a predesignated subset of sources within a specified level of distortion. The combined sampling mechanism and rate distortion code are universal in that they are devised to perform robustly without exact knowledge of the underlying joint probability distribution of the sources. In Bayesian as well as nonBayesian settings, single-letter characterizations are provided for the universal sampling rate distortion function for fixed-set sampling, independent random sampling, and memoryless random sampling. It is illustrated how these sampling mechanisms are successively better. Our achievability proofs bring forth new schemes for joint source distribution-learning and lossy compression.

We consider problems where one wishes to represent a parameter associated with a signal source - subject to a certain rate and distortion - based on the observation of a number of realizations of the source signal. By reducing these indirect vector quantization problems to a standard vector quantization one, we provide a bound to the fundamental interplay between the rate and distortion in the large-rate setting. We specialize this characterization to two particular quantization scenarios: i) the representation of the mean of a multivariate Gaussian source; and ii) the representation of the eigen-spectrum of a multivariate Gaussian source. Numerical results compare our quantization approach to an approach where one recovers the parameters from the representation of the source signals itself: in addition to revealing that the characterization is sharp in the large-rate setting, the results also show that our approach offers considerable gains.

This paper quantifies the fundamental limits of variable-length transmission of a general (possibly analog) source over a memoryless channel with noiseless feedback, under a distortion constraint. We consider excess distortion, average distortion, and guaranteed distortion (d-semifaithful codes). In contrast to the asymptotic fundamental limit, a general conclusion is that allowing variable-length codes and feedback leads to a sizable improvement in the fundamental delay-distortion tradeoff. In addition, we investigate the minimum energy required to reproduce k source samples with a given fidelity after transmission over a memoryless Gaussian channel, and we show that the required minimum energy is reduced with feedback and an average (rather than maximal) power constraint., National Science Foundation (U.S.) (Grant CCF-1016625), National Science Foundation (U.S.). Career Award (Grant CCF-1253205), National Science Foundation (U.S.). Center for Science of Information (Grant CCF-0939370)

We derive a lower bound on the differential entropy for symmetric log-concave random variable X in terms of the p-th absolute moment of X, which shows that entropy and p-th absolute moment of a symmetric log-concave random variable are comparable. We apply our bound to study the rate distortion function under distortion measure |x − x| for sources that follow a log-concave probability distribution. In particular, we establish that the difference between the rate distortion function and the Shannon lower bound is at most log(√2e) ≈ 1.9 bits, independently of r and the target distortion d. For mean-square error distortion, the difference is at most log √πe ≈ 1.55 bits, regardless of d. Our results generalize to the case of vector X. Our proof technique leverages tools from convex geometry.

Delta-Sigma modulators are often utilized to convert analog signals into digital signals. The quantization error of a Delta-Sigma modulator can be reduced by oversampling. However, oversampling increases the number of bits per time if the same number of bits are assigned to each output of the quantizer. Due to limited bandwidth, the rate-distortion relation is necessary to balance the rate and the distortion. In this paper, we analyze the relationship between the rate and the distortion of an optimal scalar Delta-Sigma modulator that minimizes the variance of the error in the output of the system connected to the Delta-Sigma modulator. Numerical examples are provided to show rate-distortion relations of the optimal Delta-Sigma modulators.

This article extends the Generalized Asypmtotic Equipartition Property of Networked Data Structures to cover the Wireless Sensor Network modelled as coloured geometric random graph (CGRG). The main techniques used to prove this result remains large deviation principles for properly defined empirical measures on CGRGs. As a motivation for this article, we apply our results to some data from Wireless Sensor Network for Monitoring Water Quality from a Lake.., Comment: 6 pages. arXiv admin note: text overlap with arXiv:1609.05252

Downsampling video coding, whereby downsampled frames are encoded, provides improved perceptual quality in rate-constrained situations. This method shows considerable advantages over other approaches, particularly in wide-spreading high-definition video formats. This paper provides a comprehensive analysis of downsampling video coding. The study proposes a spatially scalable rate-distortion (RD) model, comprising quantization-distortion and quantization-rate models, and develops an optimal encoding frame size determination framework. The proposed method achieves a gain up to 2.3 dB peak signal-to-noise ratio (PSNR) at 1 Mb/s when compared with conventional full frame size coding. The RD performance is close to the optimal scenario, in which the ideal frame size is obtained by heuristically performing downsampling coding in various allowable sizes.

Deep belief networks (DBNs) are currently the dominant technique for modeling the architectural depth of brain, and can be trained efficiently in a greedy layer-wise unsupervised learning manner. However, DBNs without a narrow hidden bottleneck typically produce redundant, continuous-valued codes and unstructured weight patterns. Taking inspiration from rate distortion (RD) theory, which encodes original data using as few bits as possible, we introduce in this paper a variant of DBN, referred to as sparse-response DBN (SR-DBN). In this approach, Kullback–Leibler divergence between the distribution of data and the equilibrium distribution defined by the building block of DBN is considered as a distortion function, and the sparse response regularization induced by L 1 -norm of codes is used to achieve a small code rate. Several experiments by extracting features from different scale image datasets show that our approach SR-DBN learns codes with small rate, extracts features at multiple levels of abstraction mimicking computations in the cortical hierarchy, and obtains more discriminative representation than PCA and several basic algorithms of DBNs.

© The Institution of Engineering and Technology 2014. The full search scheme employed in H.264/AVC significantly improves the coding performance, but it also introduces a very high computational complexity which limits the applications in resource-constrained mobile devices. In this study, the authors firstly present a discretisation total variation and orientation gradient-based hierarchical intra-prediction mode decision method for mobile video applications. By shrinking the candidate mode set in the rate-distortion optimisation (RDO) process, the proposed algorithm reduces the computational complexity and power consumption of the encoder. Furthermore, they extend the hierarchical algorithm to a complexity scalable version in which the coding complexity is measured on five levels by reserving various numbers of modes for RDO. Experimental results demonstrate that the proposed mode decision algorithm reduces the coding complexity significantly with negligible performance degradation and the proposed complexity scalable algorithm is effective and efficient for mobile video application.

By rapid advances in the development of electronic components and with declining cost of sensor network deployment, we are witnessing the amazing growth of these networks. The energy consumption constraints have imposed the importance and necessity of new concepts such as data aggregation. In this paper, a new method is proposed for aggregating the data received from the mobile point source in a sensor network. In this method called DAMORD Data Aggregation for Mobile Object using Rate-Distortion, by using RD theory and considering spatial and temporal correlation, we can reduce the load of data transmitted to base station with considering the maximum distortion that can be tolerated by the user. To the best of our knowledge, nobody has yet used the RD theory for the data aggregation of mobile object in the sensor networks. A mathematical model followed by detailed model verification demonstrated the efficiency of the proposed method under different conditions. Copyright © 2014 John Wiley & Sons, Ltd.

The latest High Efficiency Video Coding (HEVC) standard only requires 50% bit-rate of the H.264/AVC at the same perceptual quality, but with a significant encoder complexity increase. Hence, it is necessary and inevitable to develop fast HEVC encoding algorithms for its potential market adoption. In this paper, we propose a fast intra mode decision for the HEVC encoder. The overall fast intra mode decision algorithm consists of both micro- and macro-level schemes. At the micro-level, we propose the Hadamard cost-based progressive rough mode search (pRMS) to selectively check the potential modes instead of traversing all candidates (i.e., up to 35 in HEVC). Fewer effective candidates will be chosen by the pRMS for the subsequent rate-distortion optimized quantization (RDOQ) to derive the rate-distortion (R-D) optimal mode. An early RDOQ skip method is also introduced to further the complexity reduction. At the macrolevel, we introduce the early coding unit (CU) split termination if the estimated R-D cost [through aggregated R-D costs of (partial) sub-CUs] is already larger than the R-D cost of the current CU. On average, the proposed fast intra mode decision provides about 2.5 × speedup (without any platform or source code level optimization) with just a 1.0% Bjontegaard delta rate (BD-rate) increase using the HEVC common test condition. Moreover, our proposed solution also demonstrates the state-of-the-art performance in comparison with other works.

In this work, we revisit two chunk, which is the smallest video data unit for scheduling and transmission, scheduling policies that are essentially mutually exclusive but beneficial for swarm-based P2P live streaming systems. The first is content-diversified oriented (cd-oriented) policy, which regards each chunk of equal importance and schedules chunks to be sent in a near-random fashion. With this approach, peers hold different parts of stream content and contribute their available bandwidth to the system. The second is importance-first oriented (if-oriented) policy, which gives each chunk a content-dependent priority, usually in a rate-distortion (RD) sense, and first schedules the highest-priority chunk to be sent. In doing so, important chunks are more likely to be successfully received before their playback; the reconstructed video quality is thus enhanced under poor network conditions. We successfully identify a simple methodology, which operates on the data availability domain, to leverage both policies. (Data availability here means the set of data units that the user can get from its source(s)). This allows us to deploy dynamic strategy switch scheduling in practical systems to further improve the received video quality of each peer. Simulation results show that our data-availability driven dynamic strategy switch not only overcomes the drawbacks of the two individual policies but also retains the benefits of both. Most importantly, it bridges the gap between rate-distortion analysis on compressed video and P2P content delivery research.

We study formal properties of smooth max-information, a generalization of von Neumann mutual information derived from the max-relative entropy. Recent work suggests that it is a useful quantity in one-shot channel coding, quantum rate distortion theory and the physics of quantum many-body systems. Max-information can be defined in multiple ways. We demonstrate that different smoothed definitions are essentially equivalent (up to logarithmic terms in the smoothing parameters). These equivalence relations allow us to derive new chain rules for the max-information in terms of min- and max-entropies, thus extending the smooth entropy formalism to mutual information., Comment: 15 pages

I formulate an optimization framework for computing the transmission actions of streaming multi-view video content over bandwidth constrained channels. The optimization finds the schedule for sending the packetized data that maximizes the reconstruction quality of the content, for the given network bandwidth. Two prospective multi-view content representation formats are considered: 1) MVC and 2) video plus depth. In the case of each, I formulate directed graph models that characterize the interdependencies between the data units that comprise the content. For the video plus depth format, I develop a novel space-time error concealment strategy that reconstructs the missing content based on received data units from multiple views. I design multiple techniques to solve the optimization problem of interest, at varying degrees of complexity and accuracy. In conjunction, I derive spatiotemporal models of the reconstruction error for the multi-view content that I employ to reduce the computational requirements of the optimization. I study the performance of my framework via simulation experiments. Significant gains in terms of rate-distortion efficiency are demonstrated over various reference methods.

The distributed source coding theory teaches that efficient compression can be achieved by exploiting the correlation between encoded sources at the decoder only. A critical issue in this coding paradigm is how to accurately estimate the correlation statistics at the decoder. This paper proposes a novel correlation channel estimation method designed for generic layered Wyner-Ziv (WZ) coding. Driven by real-life applications, i.e., video coding, the correlation noise is assumed to be memoryless zero-mean Laplacian with a block-stationary behavior. Unlike existing methods, the proposed technique derives a novel maximum likelihood estimate of the noise scaling parameter per stationarity block. Adhering to layered coding, the derived estimate is successively refined as more stages are decoded. The Cramer-Rao lower bound (CRLB) for our estimator is derived and the mean squared error performance of the proposed successive refinement algorithm is evaluated through simulations. Apart from achieving an accuracy close to the derived CRLB, the proposed algorithm is shown to yield a WZ rate-distortion performance approaching the ideal but unrealistic scenario where the decoder knows a priori the correlation statistics. Moreover, the proposed technique has been integrated into our latest WZ video codec for wireless capsule endoscopy. Experimentation reveals that the proposed method delivers improved compression performance compared to state-of-the-art techniques, while simultaneously reducing the overall decoding complexity.

This letter presents a pixel-based motion estimation scheme assisted with the coded texture video for depth inter-prediction, in view of motion similarity between depth and texture video. The proposed scheme can achieve higher inter-prediction gain without transmitting any motion vector in the pixel-based motion estimation. Coupled with depth-texture structure similarity, the inter prediction method is further extended to an integrated prediction approach by making use of both intra and inter information. Experimental results show that our proposed method achieves superior rate-distortion performance.

While rate distortion theory compresses data under a distortion constraint, information bottleneck (IB) generalizes rate distortion theory to learning problems by replacing a distortion constraint with a constraint of relevant information. In this work, we further extend IB to multiple Markov bottlenecks (i.e., latent variables that form a Markov chain), namely Markov information bottleneck (MIB), which particularly fits better in the context of stochastic neural networks (SNNs) than the original IB. We show that Markov bottlenecks cannot simultaneously achieve their information optimality in a non-collapse MIB, and thus devise an optimality compromise. With MIB, we take the novel perspective that each layer of an SNN is a bottleneck whose learning goal is to encode relevant information in a compressed form from the data. The inference from a hidden layer to the output layer is then interpreted as a variational approximation to the layer&rsquo, s decoding of relevant information in the MIB. As a consequence of this perspective, the maximum likelihood estimate (MLE) principle in the context of SNNs becomes a special case of the variational MIB. We show that, compared to MLE, the variational MIB can encourage better information flow in SNNs in both principle and practice, and empirically improve performance in classification, adversarial robustness, and multi-modal learning in MNIST.

We write the mutual information between an input speech utterance and its reconstruction by a code-excited linear prediction (CELP) codec in terms of the mutual information between the input speech and the contributions due to the short-term predictor, the adaptive codebook, and the fixed codebook. We then show that a recently introduced quantity, the log ratio of entropy powers, can be used to estimate these mutual informations in terms of bits/sample. A key result is that for many common distributions and for Gaussian autoregressive processes, the entropy powers in the ratio can be replaced by the corresponding minimum mean squared errors. We provide examples of estimating CELP codec performance using the new results and compare these to the performance of the adaptive multirate (AMR) codec and other CELP codecs. Similar to rate distortion theory, this method only needs the input source model and the appropriate distortion measure.

Multiple description coding (MDC) is one of the source coding techniques to alleviate the problems of packet loss in the network. The decoder estimates the lost signals from received ones, based on the certain statistical correlation between descriptions. However, this correlation also leads to compression redundancy at the same time. Therefore, how to make efficient use of the introduced correlation has great importance in practical MDC approaches. In this paper, we propose a multiple description image coding scenario based on balanced multiwavelets. Two simple and effective methods to reconstruct the original image from partial descriptions are suggested. Furthermore, optimization criterion corresponding to this multiwavelet based system is provided. According to this criterion, we can choose appropriate multifilter banks to satisfy different demands. Experimental results show that the optimized multifilter banks in a simulated transform coding environment perform very well.

Modern lossy image coding systems generate a quality progressive codestream that, truncated at increasing rates, produces an image with decreasing distortion. Quality progressivity is commonly provided by an embedded quantizer that employs uniform scalar deadzone quantization (USDQ) together with a bitplane coding strategy. This paper introduces a 2-step scalar deadzone quantization (2SDQ) scheme that achieves same coding performance as that of USDQ while reducing the coding passes and the emitted symbols of the bitplane coding engine. This serves to reduce the computational costs of the codec and/or to code high dynamic range images. The main insights behind 2SDQ are the use of two quantization step sizes that approximate wavelet coefficients with more or less precision depending on their density, and a rate-distortion optimization technique that adjusts the distortion decreases produced when coding 2SDQ indexes. The integration of 2SDQ in current codecs is straightforward. The applicability and efficiency of 2SDQ are demonstrated within the framework of JPEG2000.

In High Efficiency Video Coding, the Residual Quad-Tree (RQT) coding is used to encode the prediction residual for both intra and inter Coding Units (CU) and provides improved coding gains. However, this results in much higher computational complexities. To address this problem, we develop two fast RQT algorithms for intra- and inter-prediction residual coding respectively. For intra coding, the proposed algorithm selects the best prediction mode in the rate distortion mode decision process using a Prediction Unit (PU) size-dependent fast RQT depth decision on a reduced prediction mode candidates set from the rough mode decision process. For inter coding, in addition to CU size-dependent fast RQT depth decisions, we propose a discriminant analysis-based fast depth decision algorithm to determine the best transform unit size. Experimental results show that on average, we can realise a 21.29% encoding time saving and 0.03% bit-rate reduction for intra coding, while 15% of the encoding time can be saved with a negligible coding performance loss for inter coding.

Source coding with a side information "vending machine" is a recently proposed framework in which the statistical relationship between the side information and the source, instead of being given and fixed as in the classical Wyner-Ziv problem, can be controlled by the decoder. This control action is selected by the decoder based on the message encoded by the source node. Unlike conventional settings, the message can thus carry not only information about the source to be reproduced at the decoder, but also control information aimed at improving the quality of the side information. In this paper, the analysis of the trade-offs between rate, distortion and cost associated with the control actions is extended from the previously studied point-to-point set-up to two basic multiterminal models. First, a distributed source coding model is studied, in which two encoders communicate over rate-limited links to a decoder, whose side information can be controlled. The control actions are selected by the decoder based on the messages encoded by both source nodes. For this set-up, inner bounds are derived on the rate-distortion-cost region for both cases in which the side information is available causally and non-causally at the decoder. These bounds are shown to be tight under specific assumptions, including the scenario in which the sequence observed by one of the nodes is a function of the source observed by the other and the side information is available causally at the decoder. Then, a cascade scenario in which three nodes are connected in a cascade and the last node has controllable side information, is also investigated. For this model, the rate-distortion-cost region is derived for general distortion requirements and under the assumption of causal availability of side information at the last node., 33 pages, 7 figures, submitted to IEEE Transactions on Information Theory (1st revision)

This paper investigates a lossy source coding problem in which two decoders can access their side-information respectively. The correlated sources are a product of two component correlated sources, and we exclusively investigate the case such that each component is degraded. We show the rate-distortion function for that case, and give the following observations. When the components are degraded in matched order, the rate distortion function of the product sources is equal to the sum of the component-wise rate distortion functions. On the otherhand, the former is strictly smaller than the latter when the component sources are degraded in mismatched order. The converse proof for the mismatched case is motivated by the enhancement technique used for broadcast channels. For binary Hamming and Gaussian examples, we evaluate the rate-distortion functions., Comment: 14 pages, 5 figures; A part of this paper was presented at ISIT2011 in Saint-Petersburg, Russia; v2 is the final, accepted version (to appear in IEEE IT)

This paper establishes rates attainable by several lossy schemes for coding a continuous parameter source to a specified mean-squared-error distortion based on sampling at asymptotically large rates. First, a densely sampled, spatiotemporal, stationary Gaussian source is distributively encoded. The Berger-Tung bound to the distributed rate-distortion function and three convergence theorems are used to obtain an upper bound, expressed in terms of the source spectral density, to the smallest attainable rate at asymptotically large sampling rates. The bound is tighter than that recently obtained by Kashyap Both indicate that with ideal distributed lossy coding, dense sensor networks can efficiently sense and convey a field, in contrast to the negative result obtained by Marco for encoders based on scalar quantization and Slepian-Wolf distributed lossless coding. The second scheme is transform coding with scalar coefficient quantization. A new generalized transform coding analysis, as well as the aforementioned convergence theorems, is used to find the smallest attainable rate at asymptotically large sampling rates in terms of the source spectral density and the operational rate-distortion function of the family of quantizers, which in contrast to previous analyses need not be convex. The result shows that when a transform is used, scalar quantization need not cause the poor performance found by Marco As a corollary, the final result pursues an approach, originally proposed by Berger, to show that the inverse water-pouring formula for the rate-distortion function can be attained at high sampling rates by transform coding with ideal vector quantization to encode the coefficients. Also established in the paper are relations between operational rate-distortion and distortion-rate functions for a continuous parameter source and those for the discrete parameter source that results from sampling.

In this paper, H.264/AVC primary and redundant slices interlaced multiple description video coding (PRSI-MDVC) is studied due to its high coding efficiency and ease of constructing multiple descriptions by interleaving primary and redundant slices, where the problem of optimal description generation in the rate-distortion sense is addressed. Optimal description generation requires minimization of end-to-end distortion consisting of both source coding distortion determined by quality of primary slices and channel distortion associated with redundant slice coding, subject to a rate constraint. The relevant existing works on the description generation mainly focus on the estimation of channel distortion to determine the amount of inserted redundancy (quality of redundant slices) but ignore the source distortion estimation for the quality of primary slices, thus a comprehensive end-to-end rate-distortion optimization is still unavailable. In this paper, we attack the minimization of the end-to-end distortion by fully exploring temporal coding dependency. Specifically, on one hand, a most recently developed source distortion temporal propagation model is employed to determine coding options of primary slices in the PRSI-MDVC. On the other hand, channel distortion estimation is mainly concerned with the mismatch error estimation when primary slices are lost. Unlike the existing channel distortion estimation approach under the asymptotic fine quantization assumption which is not valid in most practical cases (e.g., at low or medium coding rates), we develop a novel and more feasible estimation scheme, based on which coding parameters of the redundant slices can be better determined. Simulation results show the effectiveness of the proposed frame-level rate-distortion optimized description generation scheme compared with the relevant approaches.

In this paper we investigate encoding the bit-stream resulting from coarse Sigma-Delta quantization of finite frame expansions (i.e., overdetermined representations) of vectors. We show that for a wide range of finite-frames, including random frames and piecewise smooth frames, there exists a simple encoding algorithm—acting only on the Sigma-Delta bit stream—and an associated decoding algorithm that together yield an approximation error which decays exponentially in the number of bits used. The encoding strategy consists of applying a discrete random operator to the Sigma-Delta bit stream and assigning a binary codeword to the result. The reconstruction procedure is essentially linear and equivalent to solving a least squares minimization problem.

The JPEG2000 image compression standard is the newest still image compression standard for digital image consumer electronics, such as digital cameras. However, Tier-1 encoding in the JPEG2000 standard algorithm requires a significant amount of memory and code time. In this study, a highly efficient and low-complexity rate control algorithm for JPEG2000 is proposed to reduce the memory requirements and the computation time. A revisable threshold is developed and is used to select the passes of each code block during Tier-1 encoding. Only the selected passes are encoded during Tier-1 and Tier-2 encoding. This routine avoids the encoding of all code passes and improves the coding efficiency. The experimental results based on standard test images show that the proposed algorithm can reduce the Tier- 1 coding time by up to 86.36% and the working memory by up to 95.50% at a bit rate of 0.125 bpp in comparison to the post-compression rate distortion algorithm. The peak-signal-to-noise ratio across the images remains almost unchanged. The proposed algorithm can not only significantly improve the encoding efficiency but also maintain the image quality. Therefore, it can be applied to consumer imaging devices such as digital cameras, cameras in mobile phones and compact consumer cameras in the form of either embedded or processing software to reduce the memory and hardware requirements.

An efficient intra-prediction mode decision mechanism for H.264/AVC is presented in this research. A projection-based approach, which employs the reconstructed surrounding pixels and block content to compute the predicted block residuals (PBR), can effectively eliminate less probable modes from the computation of Rate Distortion Optimization. Both the projected vectors and corresponding predictors can be formed by shifting/adding the related data so the proposed scheme is suitable for hardware implementation. According to the PBR and coding information acquired during the encoding process, some prediction modes can also be skipped to further accelerate the intra coding. The experimental results show that the proposed scheme can effectively reduce the encoding time with slight video quality degradation and bit-rate increment.

For multi-view video plus depth (MVD) coding in 3D video (3DV) applications, texture videos and depth maps of a few reference viewpoints should be compressed and transmitted. The other intermediate virtual viewpoints are synthesized by depth-image-based rendering algorithm. The videos displayed for terminal users include the decoded reference viewpoints and the synthesized virtual viewpoints. Under a given bit rate constraint, how to choose the optimal reference viewpoints setting to obtain the best video quality of all display viewpoints is an important research issue. This paper proposes a rate distortion model based algorithm to obtain the optimal reference viewpoints selection. First, the impacts of the reference viewpoints setting on the entire quality of all display viewpoints is investigated, and then a quadratic distortion model for all display viewpoints is derived. Based on this distortion model, the optimal viewpoints selection problem is solved by computing a few model parameters. Experimental results demonstrate that the proposed distortion model estimates the actual distortion of all display viewpoints with high accuracy, and the proposed reference viewpoints selection method achieves almost the same rate distortion performance as compared to the full search method with a very low computational cost.

Randomized (dithered) quantization is a method capable of achieving white reconstruction error independent of the source. Dithered quantizers have traditionally been considered within their natural setting of uniform quantization. In this paper we extend conventional dithered quantization to nonuniform quantization, via a subterfage: dithering is performed in the companded domain. Closed form necessary conditions for optimality of the compressor and expander mappings are derived for both fixed and variable rate randomized quantization. Numerically, mappings are optimized by iteratively imposing these necessary conditions. The framework is extended to include an explicit constraint that deterministic or randomized quantizers yield reconstruction error that is uncorrelated with the source. Surprising theoretical results show direct and simple connection between the optimal constrained quantizers and their unconstrained counterparts. Numerical results for the Gaussian source provide strong evidence that the proposed constrained randomized quantizer outperforms the conventional dithered quantizer, as well as the constrained deterministic quantizer. Moreover, the proposed constrained quantizer renders the reconstruction error nearly white. In the second part of the paper, we investigate whether uncorrelated reconstruction error requires random coding to achieve asymptotic optimality. We show that for a Gaussian source, the optimal vector quantizer of asymptotically high dimension whose quantization error is uncorrelated with the source, is indeed random. Thus, random encoding in this setting of rate-distortion theory, is not merely a tool to characterize performance bounds, but a required property of quantizers that approach such bounds., Comment: submitted to IEEE Transactions on Signal Processing, 2012

Stereo interleaving video coding, in which both left and right view frames are subsampled into half size and multiplexed into one single frame before being encoded by a traditional 2-D video encoder, is an efficient encoding scenario for stereoscopic video. Many existing stereo interleaving video coding methods subsample each frame by utilizing fixed subsampling filter coefficients. Such methods are easy to implement; however, the varying property of the frame signal is ignored. By jointly considering the influences of subsampling and compression, a rate and distortion analysis about stereo interleaving video coding is proposed. The final distortion in stereo interleaving video coding is the summation of errors caused by subsampling (causing distortion between subsampling-interpolated image and the original full resolution one) and by quantization during compression. Based on the provided rate distortion analysis, a content adaptive image subsampling (CAIS) is also proposed. In CAIS, the half-size frames are generated by the optimal subsampling filters, which are calculated based on frame contents and the targeted interpolation coefficients. Experimental results demonstrate that the proposed CAIS is able to greatly improve compression efficiency of stereo interleaving video coding.

In this paper, we present a method for performing rate-distortion optimization (RDO) using a perceptual visual quality metric, the structural similarity index (SSIM), as the target of optimization. Rate-distortion optimization is widely used in modern video codecs to make various encoder decisions to optimize the rate-distortion tradeoff. Typically, the distortion measure used is either sum-of-square error or sum-of-absolute distance, both of which are convenient when used in the RDO framework but not always reflective of a perceptual visual quality. We show that SSIM can be used as the distortion metric in the RDO framework in a simple, yet effective, manner by scaling the Lagrange multiplier used in RDO based on the local variance in that region. The experimental results on the H.264/AVC reference software show that compared to traditional RDO approaches, for the same SSIM score, the proposed approach can achieve an average rate reduction of about 9% and 14% for random access and low-delay encoding configurations. At the same time, there is no significant change in the encoding runtime.

In order to solve the problem that the general image compression algorithm and the running speed based on single DSP or PC cannot satisfied the demand for high-resolution mapping real-time image compression, this paper proposed a low-complexity rata pre-allocated algorithm and an implement mode. Firstly, the multi-level integer wavelet applied to image. Secondly, depend on the principle that various subband important to image reconstruction, and the optimal rate-distortion theory, pre-allocated the actual contribution rate in the coding for the each sub-band. Lastly, when each sub-band has been coded, the coding have been truncated in real-time according to the pre-allocated rate, so the bit rate size of the compressed image has been controlled precisely. Test results show that this paper proposed algorithm which has more features than the other classical standard compression algorithm. The features include less calculation, less usage of storage space, and simple structure of the bit rate stream. The algorithm with low complexity but the same compression performance as the JPEG2000 standard is significantly better than SPIHT and other algorithms. Finally, the algorithm proposed in this paper has been realizes in real-time by many DSP and FPGA. (C) 2012 Elsevier GmbH. All rights reserved.

In this paper, an Efficient DIRECT Mode Early Decision (EDMED) algorithm is proposed for low complexity multiview video coding. Two phases are included in the proposed EDMED: 1) early decision of DIRECT mode is made before doing time-consuming motion estimation/disparity estimation, where adaptive rate-distortion (RD) cost threshold, inter-view DIRECT mode correlation and coded block pattern are jointly utilized; and 2) false rejected DIRECT mode macroblocks of the first phase are then successfully terminated based on weighted RD cost comparison between 16 16 and DIRECT modes for further complexity reduction. Experimental results show that the proposed EDMED algorithm achieves 11.76% more complexity reduction than that achieved by the state-of-the-art SDMET for the temporal views. Also, it achieves a reduction of 50.98% to 81.13% (69.15% on average) in encoding time for inter-view, which is 29.31% and 15.03% more than the encoding time reduction achieved by the state-of-the-art schemes. Meanwhile, the average Peak Signal-to-Noise Ratio (PSNR) degrades 0.05 dB and average bit rate increases by , which is negligible.

Motivated by computer networks and machine-to-machine communication applications, a bidirectional link is studied in which two nodes, Node 1 and Node 2, communicate to fulfill generally conflicting informational requirements. Node 2 is able to acquire information from the environment, for example, via access to a remote database or via sensing. Information acquisition is expensive in terms of system resources, for example, time, bandwidth and energy, and thus should be carried out efficiently by adapting the acquisition process to the needs of the application. As a result of the forward communication from Node 1 to Node 2, the latter wishes to compute some function, such as a suitable average, of the data available at Node 1 and of the data obtained from the environment. The forward link is also used by Node 1 to query Node 2 with the aim of retrieving suitable information from the environment on the backward link. The problem is formulated in the context of multi-terminal rate distortion theory and the optimal trade-off between communication rates, distortions of the information produced at the two nodes and costs for information acquisition at Node 2 is derived. The issue of robustness to possible malfunctioning of the data acquisition process at Node 2 is also investigated. The results are illustrated via an example that demonstrates the different roles played by the forward communication, namely data exchange, query and control.Copyright © 2013 John Wiley & Sons, Ltd.

This paper finds new tight finite-blocklength bounds for the best achievable lossy joint source-channel code rate, and demonstrates that joint source-channel code design brings considerable performance advantage over a separate one in the non-asymptotic regime. A joint source-channel code maps a block of $k$ source symbols onto a length$-n$ channel codeword, and the fidelity of reproduction at the receiver end is measured by the probability $\epsilon$ that the distortion exceeds a given threshold $d$. For memoryless sources and channels, it is demonstrated that the parameters of the best joint source-channel code must satisfy $nC - kR(d) \approx \sqrt{nV + k \mathcal V(d)} Q(\epsilon)$, where $C$ and $V$ are the channel capacity and channel dispersion, respectively; $R(d)$ and $\mathcal V(d)$ are the source rate-distortion and rate-dispersion functions; and $Q$ is the standard Gaussian complementary cdf. Symbol-by-symbol (uncoded) transmission is known to achieve the Shannon limit when the source and channel satisfy a certain probabilistic matching condition. In this paper we show that even when this condition is not satisfied, symbol-by-symbol transmission is, in some cases, the best known strategy in the non-asymptotic regime.

Recently, code construction approaching the rate-distortion bound of reversible data hiding has been proposed by Lin , in which the coding/decoding process needs the optimal probability distribution of marked-signals as parameters. Therefore, the efficiency and accuracy of estimating the optimal marked-signal distribution will greatly influence the speeds of encoding and decoding. In this paper, we propose a fast algorithm to solve the optimal marked-signal distribution. Furthermore, we modify the method to achieve the optimal distribution directly according to a given distortion constraint or an expected embedding rate, which makes it more practical for applications.