Search

In this paper, we study the rate-distortion (RD) optimization of the H.264 main profile encoding. Specifically, a soft decision quantization (SDQ) algorithm is developed based on the context adaptive binary arithmetic coding (CABAC) method in the H.264 main profile. Given motion prediction and quantization step sizes, the proposed SDQ algorithm is proved to achieve near-optimal SDQ for residual coding in the sense of minimizing the true RD cost when the weak adjacent block dependency utilized in CABAC is ignored for optimization. The SDQ algorithm is then used in conjunction with a general RD optimization framework to jointly design motion prediction and residual coding for H.264 main profile coding given previously coded reference frames. Experiments have been conducted based on the reference encoder JM82 of H.264 main profile. Comparative studies show that the joint design method achieves on average 10% rate reduction at the same PSNR when compared with the RD method in the H.264 main-profile reference software, with half of the reduction coming from the proposed SDQ algorithm, and 20% rate reduction at the same PSNR when compared with the RD method in the H.264 baseline-profile reference software. Index Terms--H.264 hybrid coding, rate-distortion optimization (RDO), soft decision quantization.

In this correspondence, a new digital watermarking scenario is studied, where watermarks and covertexts are correlated with each other. A necessary and sufficient condition is derived under which a watermark is first embedded, and can later be recovered with high probability at the output of a public watermark decoder after the watermarked signal is disturbed by a fixed memoryless attack channel. Interestingly, the condition also implies that the Shannon separation theorem does not hold in this scenario. Index Terms--Information embedding, random bin coding, Shannon separation theorem, Slepian-Wolf coding, typical sequences.

A typical arithmetic coder consists of three steps: range calculation, renormalization, and probability model updating. In this paper, we propose and analyze from an information theoretic point of view a greedy renormalization method, which has two components: greedy thresholding and greedy outputting. The method significantly reduces the computational complexity of the renormalization step of arithmetic coding by 1) using the greedy thresholding to minimize the number of renormalizations required to encode a sequence and 2) using the greedy outputting to minimize the number of operations within each renormalization. The method is particularly suitable for binary arithmetic coding (BAC). Two BAC algorithms based on this method are presented. The first algorithm replaces the renormalization method in the TOIS BAC [2] with the greedy renormalization method, and keeps other parts of the TOIS BAC unchanged. For binary independent and identically distributed (i.i.d.) sources with the probability of the less probable symbol ranging from 0.01-0.45, over 20% gain in speed (on average), and less than 1% loss in compression rate (in the worst case) are observed in the experiments. The second algorithm combines the greedy renormalization method with the QM-Coder. On an average, 30% gain in speed and 3% gain in compression rate are observed in the experiments. Index Terms--Arithmetic coding, computational complexity, data compression, source coding.

In this paper, we investigate the performance of grammar-based codes for sources with countably infinite alphabets. Let A denote an arbitrary class of stationary, ergodic sources with a countably infinite alphabet. It is shown that grammar-based codes can be modified so that they are universal with respect to any A if and only if there exists a universal code for A. Moreover, upper bounds on the worst case redundancies of grammar-based codes among large sets of length-n individual sequences from a countably infinite alphabet are established. Depending upon the conditions satisfied by length-n individual sequences, these bounds range from O(log log n/log n) to O(1/[log.sup.1-[alpha]] n) for some 0 < [alpha] < 1. These results complement the previous universality and redundancy results in the literature on the performance of grammar-based codes for sources with finite alphabets. Index Terms--Countably infinite alphabet, entropy, grammar-based coding, redundancy, universal data compression.

The compression performance of grammar-based codes is revisited from a new perspective. Previously, the compression performance of grammar-based codes was evaluated against that of the best arithmetic coding algorithm with finite contexts. In this correspondence, we first define semifinite-state sources and finite-order semi-Markov sources. Based on the definitions of semifinite-state sources and finite-order semi-Markov sources, and the idea of run-length encoding (RLE), we then extend traditional RLE algorithms to context-based RLE algorithms: RLE algorithms with k contexts and RLE algorithms of order k, where k is a nonnegative integer. For each individual sequence x, let [r.sup.*.sub.sr,k] (x) and [r.sup.*.sub.sr|k] (x) be the best compression rate given by RLE algorithms with k contexts and by RLE algorithms of order k, respectively. It is proved that for any x, [r.sup.*.sub.sr,k] is no greater than the best compression rate among all arithmetic coding algorithms with k contexts. Furthermore, it is shown that there exist stationary, ergodic semi-Markov sources for which the best RLE algorithms without any context outperform the best arithmetic coding algorithms with any finite number of contexts. Finally, we show that the worst case redundancies of grammar-based codes against [r.sup.*.sub.sr,k] (x) and [r.sup.*.sub.sr|k] (x) among all length. n individual sequences x from a finite alphabet are upper-bounded by [d.sub.1] log log n / log n and [d.sub.2] log log n / log n, respectively, where [d.sub.1] and [d.sub.2] are constants. This redundancy result is stronger than all previous corresponding results. Index Terms--Arithmetic coding, entropy, grammar-based coding, redundancy, renewal processes, run-length encoding (RLE), semi-Markov processes, universal data compression.

A sequence y = ([y.sub.1], ..., [y.sub.n]) is said to be a coarsening of a given finite-alphabet source sequence x = ([x.sub.1], ..., [x.sub.n]) if, for some function [phi], [y.sub.i] = [phi]([x.sub.i]) (i = 1, ..., n). In lossless refinement source coding, it is assumed that the decoder already possesses a coarsening y of a given source sequence x. It is the job of the lossless refinement source encoder to furnish the decoder with a binary codeword B(x|y) which the decoder can employ in combination with y to obtain x. We present a natural grammar-based approach for finding the binary codeword B(x|y) in two steps. In the first step of the grammar-based approach, the encoder furnishes the decoder with O([square root of n] [log.sub.2] n) code bits at the beginning of B(x|y) which tell the decoder how to build a context-free grammar [G.sub.y] which represents y. The encoder possesses a context-free grammar [G.sub.x] which represents x; in the second step of the grammar-based approach, the encoder furnishes the decoder with code bits in the rest of B(x|y) which tell the decoder how to build [G.sub.x] from [G.sub.y]. We prove that our grammar-based lossless refinement source coding scheme is universal in the sense that its maximal redundancy per sample is O(1/[log.sub.2] n) for n source samples, with respect to any finite-state lossless refinement source coding scheme. As a by-product, we provide a useful notion of the conditional entropy H([G.sub.x]|[G.sub.y]) of the grammar [G.sub.x] given the grammar [G.sub.y], which is approximately equal to the length of the codeword B (x|y). Index Terms--Conditional grammar entropy, context-free grammars, refinement source coding.

It was suggested by Battail that a good long linear code should have a weight distribution close to that of random coding, rather than a large minimum distance, and a turbo code should be also designed using a random-like criterion. In this paper, we first show that the weight distribution of a high-rate linear block code is approximately Gaussian if the code rate is close enough to one, and then proceed to construct a low-rate linear block code with approximately Gaussian weight distribution by using the turbo-coding technique. We give a sufficient condition under which the weight distribution of multicomponent turbo block (MCTB) codes (multicomponent product (MCP) codes, respectively) can approach asymptotically that of random codes, and further develop two classes of MCTB codes (MCP codes) satisfying this condition. Simulation results show that MCTB codes (MCP codes) having asymptotically Gaussian weight distribution can asymptotically approach Shannon's capacity limit. MCTB codes based on single parity-check (SPC) codes have a far poorer minimum distance than MCP codes based on SPC codes, but we show by simulation that when the bit-error rate is in the important range of [10.sup.-1][-10.sup.-5], these codes can still offer similar performance for the additive white Gaussian noise channel, as long as the code length of the SPC codes is not very short. These facts confirm in a more precise way Battail's inference about the "nonimportance" of the minimum distance for a long code. Index Terms--Gaussian approximation, linear block codes, product codes, Shannon capacity limit, turbo codes, weight distribution.

In this paper, the multilevel pattern matching (MPM) code for compression of one-dimensional (1-D) sequences is first generalized to compress two-dimensional (2-D) images, resulting in a 2-D multilevel pattern matching (MPM) code. It is shown that among all images of n pixels, the worst case redundancy of the 2-D MPM code against any finite-template-based arithmetic code is O(1/[square root of (log n)]). This result contrasts unfavorably with the fact that among all 1-D sequences of length n, the MPM code has a worst case redundancy of O(1/log n) against any finite-state arithmetic code; this is caused by the so-called 2-D boundary effect. To alleviate the 2-D boundary effect, we extend the 2-D MPM code to the case of context modeling, yielding a context-dependent 2-D MPM code. It is shown that among all images of n pixels, the context-dependent 2-D MPM code has an O(1/log n) worst case redundancy against any finite-template-based arithmetic code satisfying a mild condition; this redundancy is better than that of the 2-D MPM code without context models. Experimental results demonstrate that the context-dependent 2-D MPM code significantly outperforms the 2-D MPM code without context models for bi-level images. It is also demonstrated that, in terms of compression rates, the context-dependent 2-D MPM code performs significantly better than the progressive coding mode of JBIG1 for both textual and bi-level images, and better than or comparably to the sequential coding mode of JBIG1 and JBIG2. In addition to its excellent compression performance, the context-dependent 2-D MPM code allows progressive transmission of images. Index Terms--Context modeling, grammar-based coding, image compression, JBIG, redundancy, two-dimensional (2-D) pattern matching.

The concept of context-free grammar (CFG)-based coding is extended to the case of countable-context models, yielding context-dependent grammar (CDG)-based coding. Given a countable-context model, a greedy CDG transform is proposed. Based on this greedy CDG transform, two universal lossless data compression algorithms, an improved sequential context-dependent algorithm and a hierarchical context-dependent algorithm, are then developed. It is shown that these algorithms are all universal in the sense that they can achieve asymptotically the entropy rate of any stationary, ergodic source with a finite alphabet. Moreover, it is proved that these algorithms' worst case redundancies among all individual sequences of length n from a finite alphabet are upper-bounded by d log log n / log n, as long as the number of distinct contexts grows with the sequence length n in the order of O([n.sup.a]), where 0 < a < 1 and d are positive constants. It is further shown that for some nonstationary sources, the proposed context-dependent algorithms can achieve better expected redundancies than any existing CFG-based codes, including the Lempel-Ziv algorithm, the multilevel pattern matching algorithm, and the context-free algorithms in Part I of this series of papers. Index Terms--Arithmetic coding, context models, entropy, grammar-based coding, redundancy, string and pattern matching, universal data compression.

Giant wood spiders, Nephila maculata, are usually colored with green and yellow spots and bands across their body, but some totally dark individuals have been found. This study compared the foraging success of each of these types to discover whether or not the coloration, as reflected by ultraviolet light, affected success. The darker morph was found to have less foraging success and a lower reflecting surface across its body.

The authors discuss an innovative method of arithmetic coding that utilizes dual symbols sets to better manage image processing systems. Topics include a comparison of the new coding method to convention and zero-frequency escape arithmetic coding methods, and increased compression performance efficiency of the new method.

The authors examine light and temperature sensitivity of tin oxide for use in transistor membrane gates. Topics include light shields, pH sensitivity, and multi-structure ISFET's.

The Gold-washing data compression algorithm is an adaptive vector quantization algorithm with vector dimension n. Its asymptotic optimality has been analyzed in Parts I and Il of this series. In the paper, a redundancy problem of the Gold-washing data compression algorithm is considered. It is demonstrated that for any memoryless source with finite alphabet A and generic distribution p and for any R > 0, the redundancy of the Gold-washing data compression algorithm with dimension n (defined as the difference between the average performance of the algorithm and the distortion-rate function D(p, R) of p) is upper-bounded by [absolute value of [Delta] / [Delta]R D(p, R)] [absolute value of A] + 2[Zeta] + 4 / 2 log n / n + o (log n / n) where [Delta] / [Delta]RD(p, R) is the partial derivative of D(p, R) with respect to R, [absolute value of A] is the cardinality of A, and [Zeta] > 0 is a parameter used to control the threshold in the Gold-washing algorithm. In connection with the recent results of Zhang, Yang, and Wei on the redundancy of lossy source coding, this shows that the Goldwashing algorithm has the optimal convergence rate among all adaptive finite-state vector quantizers. Index Terms - Gold-washing algorithm, lossy data compression, rate-distortion function, redundancy, universal source coding, vector quantization.

Corresponding to any lossless codeword length function l, three universal lossy data compression schemes are presented: one is with fixed rate, another is with fixed distortion, and a third is with fixed slope. The former two universal lossy data compression schemes are the generalization of recent Yang-Kieffer's results to the general case of any lossless codeword length function l, whereas the third is new. In the case of fixed-slope [Lambda] > 0, our universal lossy data compression scheme works as follows: for any source sequence [x.sup.n] of length n, the encoder first searches for a reproduction sequence [y.sup.n] of length n which minimizes a cost function [n.sup.-1]l([y.sup.n]) + [Lambda][[Rho].sub.n]([x.sup.n], [y.sup.n]) over all reproduction sequences of length n, and then encodes [x.sup.n] into the binary codeword of length l([y.sup.n]) associated with [y.sup.n] via the lossless codeword length function l, where [[Rho].sub.n]([x.sup.n], [y.sup.n]) is the distortion per sample between [x.sup.n] and [y.sup.n]. Under some mild assumptions on the lossless codeword length function l, it is shown that when this fixed-slope data compression scheme is applied to encode a stationary, ergodic source, the resulting encoding rate per sample and the distortion per sample converge with probability one to [R.sub.[Lambda]] and [D.sub.[Lambda]], respectively, where ([D.sub.[Lambda]], [R.sub.[Lambda]]) is the point on the rate distortion curve at which the slope of the rate distortion function is -[Lambda]. This result holds particularly for the arithmetic codeword length function and Lempel-Ziv codeword length function. The main advantage of this fixed-slope universal lossy data compression scheme over the fixed-rate (fixed-distortion) universal lossy data compression scheme lies in the fact that it converts the encoding problem to a search problem through a trellis and then permits one to use some sequential search algorithms to implement it. Simulation results show that this fixed-slope universal lossy data compression scheme, combined with a suitable search algorithm, is promising. Index Terms - Arithmetic code, distortion rate (rate distortion) function, ergodic sources, fixed-slope universal source coding, Lempel-Ziv code, search algorithms, stationary, universal lossy data compression.

A new coding problem is introduced for a correlated source [Mathematical Expression Omitted]. The observer of [X.sup.n] can transmit data depending on [X.sup.n] at a prescribed rate R. Based on these data the observer of [Y.sup.n] tries to identify whether for some distortion measure [Rho] (like the Hamming distance) [n.sup.-1][Rho]([X.sup.n],[Y.sup.n]) [less than or equal to] d, a prescribed fidelity criterion. We investigate as functions of R and d the exponents of two error probabilities, the probabilities for misacceptance, and the probabilities for misrejection. In the case where [X.sup.n] and [Y.sup.n] are independent, we completely characterize the achievable region for the rate R and the exponents of two error probabilities; in the case where [X.sup.n] and [Y.sup.n] are correlated, we get some interesting partial results for the achievable region. During the process, we develop a new method for proving converses, which is called "The Inherently Typical Subset Lemma." This new method goes considerably beyond the "Entropy Characterization," the "Image Size Characterization," and its extensions. It is conceivable that this new method has a strong impact on Multiuser Information Theory. Index Terms - Identification with fidelity, misacceptance and misrejection error probabilities, multiuser information theory, rate distortion function.

The problem of redundancy of source coding with respect to a fidelity criterion is considered. For any fixed rate R > 0 and any memoryless source with finite source and reproduction alphabets and a common distribution p, the nth-order distortion redundancy [D.sub.n](R) of fixed-rate coding is defined as the minimum of the difference between the expected distortion per symbol of any block code with length n and rate R and the distortion rate function d(p, R) of the source p. It is demonstrated that for sufficiently large n, [D.sub.n](R) is equal to -([Delta]/[Delta]R)d(p, R) ln n/2n + o(ln n/n), where ([Delta]/[Delta]R)d(p, R) is the partial derivative of d(p, R) evaluated at R and assumed to exist. For any fixed distortion level d > 0 and any memoryless source p, the nth-order rate redundancy [R.sub.n](d) of coding at fixed distortion level d (or by using d-semifaithful codes) is defined as the minimum of the difference between the expected rate per symbol of any d-semifaithful code of length n and the rate-distortion function R(p, d) of p evaluated at d. It is proved that for sufficiently large n, [R.sub.n](d) is upper-bounded by ln n/n + o(ln n/n) and lower-bounded by ln n/2n + o(ln n/n). As a by-product, the lower bound of [R.sub.n](d) derived in this paper gives a positive answer to a recent conjecture proposed by Yu and Speed. Index Terms - Source coding with a fidelity criterion, rate distortion (distortion rate) function, redundancy, random-coding argument, upper joint entropy, lower mutual information.

Two versions of the gold-washing data compression algorithm, one with codebook innovation interval and the other with finitely many codebook innovations, are considered. The version of the gold-washing algorithm with codebook innovation interval k is a variant of the gold-washing algorithm such that the codebook is innovated once every k + 1 source words during the process of encoding the entire source. It is demonstrated that when this version of the gold-washing algorithm is applied to encode a stationary, [Phi]-mixing source, the expected distortion performance converges to the distortion rate function of the source as the codebook length goes to infinity. Furthermore, if the source to be encoded is a Markov source or a finite-state source, then the corresponding sample distortion performance converges almost surely to the distortion rate function. The version of the gold-washing algorithm with finitely many codebook innovations is a variant of the gold-washing algorithm in which after finitely many codebook innovations, the codebook is held fixed and reused to encode the forthcoming source sequence block by block. Similar results are shown for this version of the gold-washing algorithm. In addition, the convergence speed of the algorithm is discussed. Index Terms - Universal lossy data compression, gold-washing algorithm, distortion rate (rate distortion) function, stationary and ergodic sources, [Phi]-mixing.

The determination of complex stability constants and thermodynamic parameters for modified beta-cyclodextrins is mentioned. The results obtained show the difference in the inclusion complexes and the binding ability with cycloalkanols, camphor, and borneols.

A grammar transform is a transformation that converts any data sequence to be compressed into a grammar from which the original data sequence can be fully reconstructed. In a grammar-based code, a data sequence is first converted into a grammar by a grammar transform and then losslessly encoded. Among several recently proposed grammar transforms is the multilevel pattern matching (MPM) grammar transform. In this paper, the MPM grammar transform is first extended to the case of side information known to both the encoder and decoder, yielding a conditional MPM (CMPM) grammar transform. A new simple linear-time and space complexity algorithm is then proposed to implement the MPM and CMPM grammar transforms. Based on the CMPM grammar transform, a universal lossless data compression algorithm with side information is developed, which can achieve asymptotically the conditional entropy rate of any stationary, ergodic source pair. It is shown that the algorithm's worst case redundancy/sample against the k-context conditional empirical entropy among all individual sequences of length n is upper-bounded by c(1/log n), where c is a constant. The proposed algorithm with side information is the first in the coming family of conditional grammar-based codes, whose expected high efficiency is due to the efficiency of the corresponding unconditional codes. Index Terms--Arithmetic coding, coding with side information, entropy, grammar-based source codes, multilevel pattern matching (MPM), redundancy, string matching, universal data compression.

Throughput bounds are attained for random channel access multichannel code-division multiple-access (CDMA) systems and spread slotted Aloha systems employing multiuser receivers. It is shown that the normalized throughput of these two systems reaches 1.0 exponentially fast in the region r/K [is less than] 1, where r is the average number of simultaneous users in each channel in the random channel access multichannel CDMA system and the packet arrival rate in the spread slotted Aloha system, respectively, and K is the maximum number of users which the multiuser receiver can handle at the same time. Therefore, both of the random channel access multichannel CDMA system and the spread slotted Aloha system employing multiuser receivers can achieve perfect throughput while being stable in the region r/K = 1 - [Delta], [Delta] [is greater than] 0. The maximum throughput of the random channel access multichannel CDMA systems is found as K - [square root of (1 - (1/M))K log K - O(log K)], where M is the number of channels in the system. The maximum throughput is reached when the average number of simultaneous users is [r.sub.m] = K - [square root of (1 - (1/M))K log K] + O([square root of K/log K]). The maximum throughput of the spread slotted Aloha systems is K - [square root of K] log K - O(log K). The maximum throughput is reached when the packet arrival of Poisson distribution has the arrival rate [[Lambda].sub.m] = K - [square root of K Log K] + O ([square root of K/log K]). Index Terms--Code-division multiple access, large deviation theory, multichannel CDMA system, multiuser detection, spread slotted Aloha.

The redundancy problem of universal lossy source coding at a fixed rate level is considered. Under some condition on the single-letter distortion measure, which implies that the cardinality K of the reproduction alphabet is not greater than the cardinality J of the source alphabet, it is shown that the redundancy of universally coding memoryless sources p by nth-order block codes of rate R goes like |([Delta]/[Delta]R)d(p, R)|K ln n/2n + o(ln n/n) for all memoryless sources p except a set whose volume goes to 0 as the block length n goes to infinity, where d(p, R) denotes the distortion rate function of p. Specifically, for any sequence [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] of block codes, where [C.sub.n] is an nth-order block code at the fixed rate R, and any [element of] [is greater than] 0, the redundancy [D.sub.n]([C.sub.n], p) of [C.sub.n] for p is greater than or equal to |([Delta] / [Delta] R)d(p, R)|(K - [element of]) ln n/2n for all p satisfying some regular conditions except a set whose volume goes to 0 as n [right arrow] [infinity]. On the other hand, there exists a sequence [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] of block codes at the rate R such that for any p satisfying some regular conditions, the super limit of [D.sub.n]([C.sub.n], p)/(ln n/n) is less than or equal to |([Delta]/[Delta]R)d(P, R)|K/2. Index Terms--Data compression, lower mutual information, random coding argument, rate distortion (distortion rate) function, redundancy, universal lossy source coding, upper joint entropy.

A universal lossless data compression code called the multilevel pattern matching code (MPM code) is introduced. In processing a finite-alphabet data string of length n, the MPM code operates at O(log log n) levels sequentially. At each level, the MPM code detects matching patterns in the input data string (substrings of the data appearing in two or more nonoverlapping positions). The matching patterns detected at each level are of a fixed length which decreases by a constant factor from level to level, until this fixed length becomes one at the final level. The MPM code represents information about the matching patterns at each level as a string of tokens, with each token string encoded by an arithmetic encoder. From the concatenated encoded token strings, the decoder can reconstruct the data string via several rounds of parallel substitutions. A O(1/log n) maximal redundancy/sample upper bound is established for the MPM code with respect to any class of finite state sources of uniformly bounded complexity. We also show that the MPM code is of linear complexity in terms of time and space requirements. The results of some MPM code compression experiments are reported. Index Terms--Arithmetic coding, entropy, lossless data compression, redundancy, universal codes.

We investigate a type of lossless source code called a grammar-based code, which, in response to any input data string x over a fixed finite alphabet, selects a context-free grammar [G.sub.x] representing x in the sense that x is the unique string belonging to the language generated by [G.sub.x]. Lossless compression of x takes place indirectly via compression of the production rules of the grammar [G.sub.x]. It is shown that, subject to some mild restrictions, a grammar-based code is a universal code with respect to the family of finite-state information sources over the finite alphabet. Redundancy bounds for grammar-based codes are established. Reduction rules for designing grammar-based codes are presented. Index Terms--Chomsky hierarchy, context-free grammars, entropy, Kolmogorov complexity, lossless coding, redundancy, universal coding.

A grammar transform is a transformation that converts any data sequence to be compressed into a grammar from which the original data sequence can be fully reconstructed. In a grammar-based code, a data sequence is first converted into a grammar by a grammar transform and then losslessly encoded. In this paper, a greedy grammar transform is first presented; this grammar transform constructs sequentially a sequence of irreducible grammars from which the original data sequence can be recovered incrementally. Based on this grammar transform, three universal lossless data compression algorithms, a sequential algorithm, an improved sequential algorithm, and a hierarchical algorithm, are then developed. These algorithms combine the power of arithmetic coding with that of string matching. It is shown that these algorithms are all universal in the sense that they can achieve asymptotically the entropy rate of any stationary, ergodic source. Moreover, it is proved that their worst case redundancies among all individual sequences of length n are upper-bounded by c log log n/log n, where c is a constant. Simulation results show that the proposed algorithms outperform the Unix Compress and Gzip algorithms, which are based on LZ78 and LZ77, respectively. Index Terms--Arithmetic coding, entropy, grammar-based source codes, redundancy, string matching, universal sequential and hierarchical data compression.