Search

Estimation of the differential entropy from observations of a random variable is of great importance for a wide range of signal processing applications such as source coding, pattern recognition, hypothesis testing, and blind source separation. In this paper, we present a method for estimation of the Shannon differential entropy that accounts for embedded manifolds. The method is based on high-rate quantization theory and forms an extension of the classical nearest-neighbor entropy estimator. The estimator is consistent in the mean square sense and an upper bound on the rate of convergence of the estimator is given. Because of the close connection between compression and Shannon entropy, the proposed method has an advantage over methods estimating the Renyi entropy. Through experiments on uniformly distributed data on known manifolds and real-world speech data we show the accuracy and usefulness of our proposed method. Index Terms--Convergence rate, manifolds, nearest-neighbor distance, Shannon differential entropy.

To facilitate real-time audiovisual communication through the Internet, forward error correction (FEC) and multiple description coding (MDC) can be used as low-delay packet-loss recovery techniques. We use both a Gilbert channel model and data obtained from real IP connections to compare the rate-distortion performance of different variants of FEC and MDC. Using identical overall rates with stringent delay constraints, we find that side-distortion optimized MDC generally performs better than Reed-Solomon-based FEC. If the channel condition is known from feedback, then channel-optimized MDC can be used to exploit this information, resulting in significantly improved performance. Our results confirm that two-independent-channel transmission is preferred to single-channel transmission, both for FEC and MDC. Index Terms--Channel coding, error correction, information theory, multimedia communication, source coding.

We introduce multivariate block polar quantization (MBPQ). MBPQ minimizes the weighted squared-error distortion for a set of complex variables representing one block of a signal under a resolution constraint for the entire block. MBPQ performs below the lower bound for classical bivariate quantization, both for Gaussian complex variables and for complex variables found from sinusoidal analysis of audio data. Still, it is of similar complexity as traditional polar quantizers. In the case of audio data, we found a performance gain of about 2.5 dB over the best performing conventional resolution-constrained polar quantization (an extension of unrestricted polar quantization). Index Terms--Audio coding, high rate, polar quantization, rate-distortion theory, scalar quantization.